Runs and results

Start a run

POST /v1/scrapers/{slug}/runs

Body field	Type	What it does
input	object	The scraper's input. Each scraper page lists the fields.
maxCostUsd	number	The most this run may cost. It is held from your balance while the run works, and the run stops when it reaches it.
wait	integer, up to 60	Seconds to wait for the run to finish before answering.
webhookUrl	string	An https address we call when the run is done.

Send an Idempotency-Key header to make retries safe: the same key within 24 hours returns the same run instead of starting a new one.

The answer is 200 with the finished run when it ends within wait, or 202 with { "id": "...", "status": "running" }.

Check a run

GET /v1/runs/{id} returns the status (queued, running, settling, done or failed), the row count and the cost so far. Rows become readable when the status is done, once their cost is settled.

Read the rows

GET /v1/runs/{id}/items?offset=0&limit=500&format=json

• format is json, ndjson or csv.
• limit is at most 500; page with offset.
• Reading rows is free.

Each row has a rowType. Rows of the scraper's main type are charged. Error rows (an input could not be read) and coverage rows (how much of a source was read, and why it stopped) are free.

Webhooks

When you pass webhookUrl, we POST { id, status, itemCount, chargedUsd } to it when the run ends, signed with an X-Arcadium-Signature header (HMAC with your account's webhook secret). Failed deliveries are retried after 1 minute, 10 minutes and 1 hour.

What you pay

You pay the scraper's price for each delivered row, plus its start fee when the run delivers rows. A run that returns rows costs at least $0.005, start fee included. Failed runs, empty runs and duplicate rows cost $0. Prices are on every scraper page and on Pricing.

Errors

Status	When
400	The input does not match the scraper's schema, or a field is not allowed. The message names the fields.
401	Missing or invalid key.
402	Your balance is lower than the run's hold.
404	No scraper or run with that ID.
429	Too many requests or runs at once. Retry after the time in Retry-After.
5xx	Our side failed. Retry with the same Idempotency-Key.