social-media Scenario 4 — Forum Thread — Extract Nested Comments
Written by RTILA X Engineering Team
Intermediate ~6 min
Prerequisites: social-media/1 social-media/2
What You'll Learn
Handle infinite-scroll feeds on social media
Real-World Use Case
Archiving a complete timeline
LIVE DEMO
Best practices for web scraping in 2025?
EXPECTED OUTPUT
RTILA X PROJECT JSON
{
"name": "Extract_Forum_Thread",
"settings": {
"urls": [
"https://learn.rtila.com/scenarios/social-media/4"
]
},
"datasets": {
"comments": {
"item_selector": "css=.forum-comment",
"properties": [
{
"name": "author",
"type": "text",
"selector": "css=.comment-author"
},
{
"name": "time",
"type": "text",
"selector": "css=.comment-time"
},
{
"name": "text",
"type": "text",
"selector": "css=.comment-text"
},
{
"name": "replies",
"type": "list",
"selector": "css=.forum-reply",
"properties": [
{
"name": "author",
"type": "text",
"selector": "css=.reply-author"
},
{
"name": "time",
"type": "text",
"selector": "css=.reply-time"
},
{
"name": "text",
"type": "text",
"selector": "css=.reply-text"
}
]
}
]
}
},
"commands": [
{
"command": "wait_for_selector",
"params": {
"selector": "css=.forum-comment"
}
},
{
"command": "extract_data",
"params": {
"dataset": "comments"
}
}
]
}
Nested comment structures are represented with a list property inside the comment dataset. Each reply has its own author, timestamp, and text selectors.
Did you complete this scenario?
Ready to run this automation?
Copy the project JSON above, open RTILA X, create a new project, and paste it.
This scenario covers 2 RTILA X commands and is referenced in 2 learning articles.
Continue Learning
Was this helpful?
Thank you for your feedback!
Comment body 1 discussing the thread topic.
Comment body 2 discussing the thread topic.
Comment body 3 discussing the thread topic.
Comment body 4 discussing the thread topic.
Comment body 5 discussing the thread topic.
Comment body 6 discussing the thread topic.
Comment body 7 discussing the thread topic.
Comment body 8 discussing the thread topic.