Skip to main content

basic-navigation Scenario 2 — Scrape Top 30 Headlines with URLs

Written by RTILA X Engineering Team

Beginner ~4 min

What You'll Learn

Scrape headlines and their URLs using attribute extraction

Real-World Use Case

Building a news aggregator

LIVE DEMO

News List

Thirty article entries with stable heading, link, and container selectors.

The Future of Edge Computing in 2025

Read Article

How Remote Teams Boost Productivity

Read Article

CRISPR Gene Editing Breakthroughs

Read Article

10 Habits for Better Sleep

Read Article

Hidden Gems of Southeast Asia

Read Article

WebAssembly Adoption Trends

Read Article

The Rise of Headless E-Commerce

Read Article

Quantum Computing Progress Report

Read Article

Understanding Blood Sugar Control

Read Article

Island Hopping in Greece

Read Article

Progress on Autonomous Vehicles

Read Article

Cultivating a Learning Culture

Read Article

Battery Technology in 2025

Read Article

The Benefits of Intermittent Fasting

Read Article

Backpacking the Dolomites

Read Article

The Evolution of Developer Tools

Read Article

Sustainable Business Models

Read Article

Understanding Neuroplasticity

Read Article

Everyday Strategies for Mental Wellness

Read Article

Exploring the Camino de Santiago

Read Article

Introduction to Data Mesh

Read Article

Outsourcing Without Losing Quality

Read Article

Space Exploration in 2025

Read Article

The Gut Microbiome and Health

Read Article

Urban Exploration in Tokyo

Read Article

MLOps: From Idea to Production

Read Article

Building Resilient Supply Chains

Read Article

Understanding the Immune System

Read Article

Road Tripping the Norwegian Fjords

Read Article

Retrieval-Augmented Generation Explained

Read Article
EXPECTED OUTPUT

Headlines and URLs

title url
The Future of Edge Computing in 2025 https://learn.rtila.com/articles/1
How Remote Teams Boost Productivity https://learn.rtila.com/articles/2
CRISPR Gene Editing Breakthroughs https://learn.rtila.com/articles/3
10 Habits for Better Sleep https://learn.rtila.com/articles/4
Hidden Gems of Southeast Asia https://learn.rtila.com/articles/5
WebAssembly Adoption Trends https://learn.rtila.com/articles/6
The Rise of Headless E-Commerce https://learn.rtila.com/articles/7
Quantum Computing Progress Report https://learn.rtila.com/articles/8
Understanding Blood Sugar Control https://learn.rtila.com/articles/9
Island Hopping in Greece https://learn.rtila.com/articles/10
Progress on Autonomous Vehicles https://learn.rtila.com/articles/11
Cultivating a Learning Culture https://learn.rtila.com/articles/12
Battery Technology in 2025 https://learn.rtila.com/articles/13
The Benefits of Intermittent Fasting https://learn.rtila.com/articles/14
Backpacking the Dolomites https://learn.rtila.com/articles/15
The Evolution of Developer Tools https://learn.rtila.com/articles/16
Sustainable Business Models https://learn.rtila.com/articles/17
Understanding Neuroplasticity https://learn.rtila.com/articles/18
Everyday Strategies for Mental Wellness https://learn.rtila.com/articles/19
Exploring the Camino de Santiago https://learn.rtila.com/articles/20
Introduction to Data Mesh https://learn.rtila.com/articles/21
Outsourcing Without Losing Quality https://learn.rtila.com/articles/22
Space Exploration in 2025 https://learn.rtila.com/articles/23
The Gut Microbiome and Health https://learn.rtila.com/articles/24
Urban Exploration in Tokyo https://learn.rtila.com/articles/25
MLOps: From Idea to Production https://learn.rtila.com/articles/26
Building Resilient Supply Chains https://learn.rtila.com/articles/27
Understanding the Immune System https://learn.rtila.com/articles/28
Road Tripping the Norwegian Fjords https://learn.rtila.com/articles/29
Retrieval-Augmented Generation Explained https://learn.rtila.com/articles/30
RTILA X PROJECT JSON

RTILA X Automation Project

scrape-headlines-urls.json
    {
  "name": "Scrape_Top_30_Headlines_With_URLs",
  "settings": {
    "urls": [
      "https://learn.rtila.com/scenarios/basic-navigation/2"
    ],
    "headless": false,
    "navigation_timeout": 30000,
    "max_concurrent_workers": 1,
    "humanoidEnabled": true,
    "humanoidSensitivity": "medium"
  },
  "datasets": {
    "articles": {
      "item_selector": "css=div.article-item",
      "properties": [
        {
          "name": "title",
          "type": "text",
          "selector": "css=h3.article-title"
        },
        {
          "name": "url",
          "type": "attribute",
          "selector": "css=a.article-link",
          "attribute": "href"
        }
      ]
    }
  },
  "commands": [
    {
      "command": "goto",
      "params": {
        "url": "https://learn.rtila.com/scenarios/basic-navigation/2"
      }
    },
    {
      "command": "wait_for_selector",
      "params": {
        "selector": "css=div.article-item"
      }
    },
    {
      "command": "extract_data",
      "params": {
        "dataset": "articles"
      }
    }
  ]
}
  

RTILA X Concepts

This scenario combines two extraction types in one dataset: text for the article title and attribute for the link href. It also demonstrates selecting a repeated parent with item_selector.

Did you complete this scenario?

Ready to run this automation?

Copy the project JSON above, open RTILA X, create a new project, and paste it.

This scenario covers 2 RTILA X commands and is referenced in 2 learning articles.

Continue Learning

Was this helpful?