Skip to main content

variables Scenario 3 — Set Base URL Variable and Navigate Subpages

Written by RTILA X Engineering Team

Intermediate ~6 min
Prerequisites: variables/1 variables/2

What You'll Learn

Construct dynamic URLs using base and path variables

Real-World Use Case

Building a domain-independent scraper

LIVE DEMO

Subpage Content

This content appears when navigating to a subpage.

EXPECTED OUTPUT

RTILA X PROJECT JSON 

    {
  "name": "Base_URL_Variable_Navigation",
  "settings": {
    "urls": [
      "https://learn.rtila.com/scenarios/variables/3"
    ],
    "headless": false,
    "navigation_timeout": 30000,
    "max_concurrent_workers": 1,
    "humanoidEnabled": true,
    "humanoidSensitivity": "medium"
  },
  "datasets": {
    "subpage_data": {
      "item_selector": "css=.subpage-content h2",
      "properties": [
        {
          "name": "content",
          "type": "text",
          "selector": "css=self"
        }
      ]
    }
  },
  "commands": [
    {
      "command": "goto",
      "params": {
        "url": "https://learn.rtila.com/scenarios/variables/3"
      }
    },
    {
      "command": "set_variable",
      "params": {
        "name": "base_url",
        "value": "https://learn.rtila.com/scenarios/variables"
      }
    },
    {
      "command": "goto",
      "params": {
        "url": "${base_url}/page1"
      }
    },
    {
      "command": "wait_for_selector",
      "params": {
        "selector": "css=.subpage-content h2"
      }
    },
    {
      "command": "extract_data",
      "params": {
        "dataset": "subpage_data"
      }
    },
    {
      "command": "goto",
      "params": {
        "url": "${base_url}/page2"
      }
    },
    {
      "command": "wait_for_selector",
      "params": {
        "selector": "css=.subpage-content h2"
      }
    },
    {
      "command": "extract_data",
      "params": {
        "dataset": "subpage_data"
      }
    },
    {
      "command": "goto",
      "params": {
        "url": "${base_url}/page3"
      }
    },
    {
      "command": "wait_for_selector",
      "params": {
        "selector": "css=.subpage-content h2"
      }
    },
    {
      "command": "extract_data",
      "params": {
        "dataset": "subpage_data"
      }
    }
  ]
}

Using set_variable to store a base URL makes the navigation reusable. Each subpage URL is created by concatenating the base URL with a page slug.

Did you complete this scenario?

Ready to run this automation?

Copy the project JSON above, open RTILA X, create a new project, and paste it.

Download RTILA X Free → View setup guide →

This scenario covers 2 RTILA X commands and is referenced in 1 learning article.

Continue Learning

Was this helpful?