Sources
Upload domains
Crawl a domain into the project knowledge base
POST
Adds a domain crawl source and queues discovery/indexing. Returns
201 Created.
Authentication
Project API key for this organization and project. Format:
bot_live_…Path parameters
Organization ID. Must match the API key.
Project ID. Must match the API key.
Body
Domain or site root to crawl (for example
https://example.com).sitemap or deep_crawl.Maximum pages to crawl. Range:
1–1000.Maximum crawl depth. Range:
1–10.Whitelist regex/glob patterns; only matching URLs are crawled.
Patterns to exclude from the crawl.
Automatic recrawl interval in days (
1–365).Crawler engine:
crawl4ai or apify.Response
Success message confirming the domain crawl was queued.data is typically null.
Errors
Missing, invalid, or revoked API key.
API key is not authorized, or link quota exceeded.

