Skip to main content

loops Scenario 7 — Iterate Over Dataset Items for Detail Pages

Written by RTILA X Engineering Team

Advanced ~10 min
Prerequisites: loops/3 loops/4 loops/5

What You'll Learn

Perform two-phase scraping across list and detail pages

Real-World Use Case

Building a comprehensive product database

LIVE DEMO

Dataset Extraction Demo

This section demonstrates iterating over extracted dataset items and visiting each detail page.

Product 101 Details

Premium wireless over-ear headphones.

$299.99

Product 102 Details

High-speed mechanical keyboard.

$89.99

Product 103 Details

Portable Bluetooth speaker.

$149.99

Product 104 Details

4K OLED monitor with HDR.

$499.99

Product 105 Details

Ergonomic wireless mouse.

$34.99
EXPECTED OUTPUT

RTILA X PROJECT JSON 

    {
  "name": "Dataset_Detail_Pages",
  "settings": {
    "urls": [
      "https://learn.rtila.com/scenarios/loops/7"
    ],
    "headless": false,
    "navigation_timeout": 30000,
    "max_concurrent_workers": 1,
    "humanoidEnabled": true,
    "humanoidSensitivity": "medium"
  },
  "datasets": {
    "items": {
      "item_selector": "css=.detail-item",
      "properties": [
        {
          "name": "title",
          "type": "text",
          "selector": "css=.detail-title"
        },
        {
          "name": "description",
          "type": "text",
          "selector": "css=.detail-description"
        },
        {
          "name": "price",
          "type": "text",
          "selector": "css=.detail-price"
        }
      ]
    },
    "details": {
      "item_selector": "css=article",
      "properties": [
        {
          "name": "full_text",
          "type": "text",
          "selector": "css=p"
        }
      ]
    }
  },
  "commands": [
    {
      "command": "goto",
      "params": {
        "url": "https://learn.rtila.com/scenarios/loops/7"
      }
    },
    {
      "command": "wait_for_selector",
      "params": {
        "selector": "css=.detail-item"
      }
    },
    {
      "command": "extract_data",
      "params": {
        "dataset": "items"
      }
    },
    {
      "command": "for_each",
      "params": {
        "source_type": "dataset",
        "dataset": "items",
        "as": "item",
        "do": [
          {
            "command": "goto",
            "params": {
              "url": "${item.url}"
            }
          },
          {
            "command": "wait_for_selector",
            "params": {
              "selector": "css=article"
            }
          },
          {
            "command": "extract_data",
            "params": {
              "dataset": "details"
            }
          }
        ]
      }
    }
  ]
}

RTILA X Concepts

A second for_each can iterate over a previously extracted dataset. Each row provides values for navigation and extraction, enabling two-phase scraping patterns.

Did you complete this scenario?

Ready to run this automation?

Copy the project JSON above, open RTILA X, create a new project, and paste it.

Download RTILA X Free → View setup guide →

This scenario covers 3 RTILA X commands and is referenced in 2 learning articles.

Continue Learning

Was this helpful?