Skip to main content

variables Scenario 4 — API Pagination with Token and Page Variables

Written by RTILA X Engineering Team

Intermediate ~6 min
Prerequisites: variables/1 variables/2

What You'll Learn

Manipulate lists with push and pop

Real-World Use Case

Collecting URLs during a crawl

LIVE DEMO

1AlphaA1
2BetaB2
3GammaC3
4DeltaD4
5EpsilonE5
Page 1 of 5
EXPECTED OUTPUT

RTILA X PROJECT JSON 

    {
  "name": "API_Pagination_Token_And_Page",
  "settings": {
    "urls": [
      "https://learn.rtila.com/scenarios/variables/4"
    ],
    "headless": false,
    "navigation_timeout": 30000,
    "max_concurrent_workers": 1,
    "humanoidEnabled": true,
    "humanoidSensitivity": "medium",
    "variables": [
      {
        "name": "api_token",
        "is_list": false,
        "value": "demo-token-abc123"
      },
      {
        "name": "page_num",
        "is_list": false,
        "value": "1"
      }
    ]
  },
  "datasets": {
    "api_page_data": {
      "item_selector": "css=table.api-page-data tbody tr",
      "properties": [
        {
          "name": "id",
          "type": "text",
          "selector": "css=td:nth-child(1)"
        },
        {
          "name": "name",
          "type": "text",
          "selector": "css=td:nth-child(2)"
        },
        {
          "name": "value",
          "type": "text",
          "selector": "css=td:nth-child(3)"
        }
      ]
    }
  },
  "commands": [
    {
      "command": "goto",
      "params": {
        "url": "https://api.example.com/data?token=${api_token}&page=${page_num}"
      }
    },
    {
      "command": "wait_for_selector",
      "params": {
        "selector": "css=.api-page-data",
        "iframe": "",
        "timeout": 30000
      }
    },
    {
      "command": "extract_data",
      "params": {
        "dataset": "api_page_data"
      }
    },
    {
      "command": "set_variable",
      "params": {
        "name": "page_num",
        "value": "2"
      }
    },
    {
      "command": "repeat",
      "params": {
        "times": 4,
        "do": [
          {
            "command": "goto",
            "params": {
              "url": "https://api.example.com/data?token=${api_token}&page=${page_num}"
            }
          },
          {
            "command": "wait_for_selector",
            "params": {
              "selector": "css=.api-page-data",
              "iframe": "",
              "timeout": 30000
            }
          },
          {
            "command": "extract_data",
            "params": {
              "dataset": "api_page_data"
            }
          },
          {
            "command": "math_operation",
            "params": {
              "left_operand": "${page_num}",
              "operator": "+",
              "right_operand": "1",
              "output_variable": "page_num"
            }
          }
        ]
      }
    }
  ]
}

API pagination often uses auth tokens and page numbers. The variables are inserted into each request URL and updated after every page extraction.

Did you complete this scenario?

Ready to run this automation?

Copy the project JSON above, open RTILA X, create a new project, and paste it.

Download RTILA X Free → View setup guide →

This scenario covers 1 RTILA X command and is referenced in 2 learning articles.

Continue Learning

Was this helpful?