Skip to main content

loops Scenario 5 — Iterate Category Links and Extract Products

Written by RTILA X Engineering Team

Intermediate ~6 min
Prerequisites: loops/1 loops/2

What You'll Learn

Skip iterations with continue

Real-World Use Case

Filtering out unwanted items in a loop

LIVE DEMO

Wireless Noise-Cancelling Headphones

$249.99

4K Ultra HD Monitor

$499.99

Mechanical Gaming Keyboard

$129.99

USB-C Docking Station

$159.99

Portable SSD 1TB

$99.99

Smart Home Hub

$79.99
EXPECTED OUTPUT

RTILA X PROJECT JSON 

    {
  "name": "Iterate_Category_Links",
  "settings": {
    "urls": [
      "https://learn.rtila.com/scenarios/loops/5"
    ],
    "headless": false,
    "navigation_timeout": 30000,
    "max_concurrent_workers": 1,
    "humanoidEnabled": true,
    "humanoidSensitivity": "medium"
  },
  "datasets": {
    "categories": {
      "item_selector": "css=a.category-link",
      "properties": [
        {
          "name": "name",
          "type": "text",
          "selector": "css=self"
        },
        {
          "name": "href",
          "type": "attribute",
          "selector": "css=self",
          "attribute": "href"
        }
      ]
    },
    "category_products": {
      "item_selector": "css=.category-product",
      "properties": [
        {
          "name": "name",
          "type": "text",
          "selector": "css=.cat-product-name"
        },
        {
          "name": "price",
          "type": "text",
          "selector": "css=.cat-product-price"
        }
      ]
    }
  },
  "commands": [
    {
      "command": "goto",
      "params": {
        "url": "https://learn.rtila.com/scenarios/loops/5"
      }
    },
    {
      "command": "wait_for_selector",
      "params": {
        "selector": "css=.category-link"
      }
    },
    {
      "command": "extract_data",
      "params": {
        "dataset": "categories"
      }
    },
    {
      "command": "for_each",
      "params": {
        "source_type": "dataset",
        "dataset": "categories",
        "as": "category",
        "do": [
          {
            "command": "goto",
            "params": {
              "url": "${category.href}"
            }
          },
          {
            "command": "wait_for_selector",
            "params": {
              "selector": "css=.category-product"
            }
          },
          {
            "command": "extract_data",
            "params": {
              "dataset": "category_products"
            }
          },
          {
            "command": "go_back",
            "params": {}
          }
        ]
      }
    }
  ]
}

RTILA X Concepts

This workflow extracts a dataset of category links and then uses a for_each over that dataset. Each category page is opened, products are scraped, and the automation returns to the category list with go_back.

Did you complete this scenario?

Ready to run this automation?

Copy the project JSON above, open RTILA X, create a new project, and paste it.

Download RTILA X Free → View setup guide →

This scenario covers 2 RTILA X commands and is referenced in 1 learning article.

Continue Learning

Was this helpful?