Skip to main content
Crawl

Authorizations

Authorization
string
header
required

Send an active organization API key in the Authorization: Bearer header.

Headers

Idempotency-Key
string | null

Body

application/json
url
string
required
Required string length: 1 - 8192
crawlEntireDomain
boolean
default:false
excludePaths
string[]
Maximum array length: 20
ignoreQueryParameters
boolean
default:false
includePaths
string[]
Maximum array length: 20
includeSubdomains
boolean
default:false
limit
integer
default:20
Required range: 1 <= x <= 100
maxDepth
integer
default:3
Required range: 0 <= x <= 10
scrapeOptions
ScrapeOptions · object

Everything a caller can ask for on a single page.

sitemap
enum<string>
default:include
Available options:
include,
skip,
only

Response

Successful Response

data
CollectionStatus · object | null
required
message
string | null
required
meta
Meta · object | null
required
request_id
string | null
required
success
boolean
default:true
required