BotAnon &mdash; API Documentation

Proxy API
Endpoint
Authentication
Parameters
Custom Headers
Response Format
Response Field Reference
Error Codes
Proxy Chain
Caching
Rate Limits
Code Examples
Google Maps API
Search Places
Place Lookup
Crawl (Queue / Instant)
Stats
Twitter/X Search API
Search Tweets
Manage Accounts

Endpoint

GET https://www.botanon.com/api/fetch?url=...&apikey=...

POST https://www.botanon.com/api/fetch (JSON body)

Both methods accept the same parameters. Use POST when sending custom headers or complex POST data.

Authentication

Every request requires an apikey parameter. Contact [email protected] to get a key.

GET /api/fetch?url=https://example.com&apikey=YOUR_KEY

Parameters

Required

Parameter	Description
url	Target URL to fetch (http:// or https://)
apikey	Your API key

Request Options

Parameter	Default	Description
method	GET	HTTP method on target: GET or POST
post_data		Request body for POST requests
post_type	form	POST encoding: form or json
headers		Custom headers to forward to target (object, POST JSON only)
ua	Chrome 131	User-Agent, or random for random modern browser UA
timeout	30	Timeout in seconds (max 120)
tag		Label for log filtering (e.g. seo_audit)
originator_ip		End-user IP for per-user rate limiting

Proxy Options

Parameter	Default	Description
proxy	auto	Force proxy: eris, zyte, scrapeowl, scrapedo, residential, browser, browser_fetch, scamadviser_cf, trustpilot_cf, none. Comma-separate for a chain: browser_fetch,scrapeowl.
render_js	0	JS rendering via ScrapeOwl/Zyte (cheaper than browser)
premium	0	ScrapeOwl premium proxies
browser	0	Full headless browser via Cloudflare
country		Geo-target: ISO code (e.g. US, GB)

Cache Options

Parameter	Default	Description
refresh	0	Bypass cache, fetch fresh
cache		Override cache TTL in days (1–365)
compress	0	Gzip-compressed response

Cost: Default (free) → render_js=1 (API credits) → premium=1 (2x credits) → browser=1 (most expensive). Escalate only when needed.

Custom Headers

Forward API keys, auth tokens, etc. to the target via POST JSON body:

cURL

curl -s -X POST https://www.botanon.com/api/fetch \
  -H 'Content-Type: application/json' \
  -d '{
    "url": "https://api.bing.microsoft.com/v7.0/search?q=test",
    "apikey": "YOUR_KEY",
    "headers": {"Ocp-Apim-Subscription-Key": "YOUR_BING_KEY"},
    "tag": "bing"
  }' | jq .

Python

resp = requests.post("https://www.botanon.com/api/fetch", json={
    "url": "https://api.bing.microsoft.com/v7.0/search?q=test",
    "apikey": "YOUR_KEY",
    "headers": {"Ocp-Apim-Subscription-Key": "YOUR_BING_KEY"},
    "tag": "bing",
})

PHP

$ch = curl_init('https://www.botanon.com/api/fetch');
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_POST           => true,
    CURLOPT_HTTPHEADER     => ['Content-Type: application/json'],
    CURLOPT_POSTFIELDS     => json_encode([
        'url'     => 'https://api.bing.microsoft.com/v7.0/search?q=test',
        'apikey'  => 'YOUR_KEY',
        'headers' => ['Ocp-Apim-Subscription-Key' => 'YOUR_BING_KEY'],
        'tag'     => 'bing',
    ]),
]);

Supported by all drivers: Eris (via x-fwd- prefix), Zyte, ScrapeOwl, Residential, Direct. Not supported by Browser (CF headless).

Response Format

Success (HTTP 200)

{
  "success": true,
  "request_id": "a7c3e9f2-4b1d-4e8a-9f2c-1a2b3c4d5e6f",
  "url": "https://example.com",
  "final_url": "https://www.example.com/",
  "http_code": 200,
  "body": "<!doctype html>\n<html>...",
  "body_size": 45230,
  "body_compressed_size": 8102,
  "headers": {
    "Content-Type": "text/html; charset=UTF-8",
    "Server": "nginx"
  },
  "timing": {
    "total_ms": 1230.5,
    "connect_ms": 150.2,
    "ssl_ms": 80.1,
    "transfer_ms": 320.0
  },
  "proxy": {
    "driver": "eris",
    "name": "Eris Worker",
    "proxy_id": 1
  },
  "retries": 0,
  "cached": false,
  "cache_expires": "2026-04-18 14:30:00",
  "compressed": false,
  "customer": {
    "requests_today": 142,
    "success_today": 140,
    "failures_today": 2,
    "requests_this_month": 4235
  }
}

Failure (HTTP 502)

{
  "success": false,
  "error_code": "ALL_PROXIES_FAILED",
  "error": "All proxies failed",
  "url": "https://blocked-site.com",
  "http_code": 403,
  "last_proxy_error": "HTTP 403",
  "last_proxy_driver": "residential",
  "retries": 3,
  "proxies_tried": ["eris", "zyte", "scrapeowl", "residential"]
}

Rate Limited (HTTP 429)

{
  "success": false,
  "error_code": "RATE_LIMIT_CUSTOMER",
  "error": "Customer rate limit exceeded: 1001/1000 requests per hour",
  "limit": 1000,
  "current": 1001,
  "retry_after": 60
}

Response Field Reference

Success Fields

Field	Type	Description
success	bool	Always true
request_id	string	Unique UUID for log correlation
url	string	The URL you requested
final_url	string	URL after redirects
http_code	int	HTTP status from target (200, 301, 404, etc.)
body	string	Full response body (HTML, JSON as string, text)
body_size	int	Body size in bytes
body_compressed_size	int	Body size after gzip
headers	object	Response headers from target
timing.total_ms	float	Total request time (ms)
timing.connect_ms	float	TCP connection time
timing.ssl_ms	float	SSL handshake time
timing.transfer_ms	float	Time to first byte
proxy.driver	string	Proxy used: eris, zyte, scrapeowl, residential, browser, none
proxy.name	string	Human-readable proxy name
proxy.proxy_id	int	Internal proxy ID
retries	int	Proxies that failed before success (0 = first try)
cached	bool	true if from cache (timing = 0)
cache_expires	string	Cache expiry datetime
compressed	bool	true if gzip-compressed
customer.requests_today	int	Your requests today
customer.success_today	int	Successful today
customer.failures_today	int	Failed today
customer.requests_this_month	int	Requests this month

Failure Fields

Field	Type	Description
success	bool	Always false
error_code	string	Machine-readable error code
error	string	Human-readable message
http_code	int	HTTP code from last proxy (0 if connection failed)
last_proxy_error	string	Last proxy's error message
last_proxy_driver	string	Last proxy driver tried
retries	int	Total proxies tried
proxies_tried	array	Drivers attempted, in order
limit	int	Rate limit threshold (429 only)
current	int	Current count in window (429 only)
retry_after	int	Seconds to wait (429 only)

JSON body: When the target returns JSON, the body field is a JSON string. Parse it: json.loads(data["body"]) (Python), json_decode($data['body'], true) (PHP), JSON.parse(data.body) (JS).

Error Codes

Code	HTTP	Meaning
MISSING_URL	400	No url parameter
MISSING_APIKEY	401	No apikey parameter
INVALID_APIKEY	403	Key not found or customer inactive
INVALID_URL	400	Malformed, not http/https, or private IP
INVALID_TIMEOUT	400	Exceeds customer max
INVALID_METHOD	400	Not GET or POST
INVALID_PROXY	400	Unknown proxy driver
RATE_LIMIT_CUSTOMER	429	Hourly limit exceeded
RATE_LIMIT_CALLER_IP	429	Caller IP limit exceeded
RATE_LIMIT_ORIGINATOR_IP	429	Originator IP limit exceeded
RATE_LIMIT_DAILY	429	Daily limit exceeded
ALL_PROXIES_FAILED	502	Every proxy failed
PROXY_ERROR	502	Proxy returned non-2xx
INTERNAL_ERROR	500	Server error

Proxy Chain

Default failover chain (configurable per-customer):

#	Driver	Type	Best For
1	eris	CF Worker	Most sites, free, fast
2	zyte	Premium API	Anti-bot, CAPTCHAs
3	scrapeowl	Premium API	JS rendering, premium proxies
4	residential	Residential IPs	Datacenter-blocked sites

Special proxies (explicit only)

These aren't in any default chain — opt in by passing proxy=X. Chain with proxy=X,scrapeowl to fall back on failure.

Driver	Purpose	Example
browser	Cloudflare Browser Rendering API (managed, paid)	`proxy=browser`
browser_fetch	Generic CF Puppeteer worker — renders any JS-heavy URL, returns HTML. ~25s cold start, use timeout=45.	`proxy=browser_fetch,scrapeowl`
scrapedo	Scrape.do API (test-only). Supports render_js (headless browser), premium (residential/mobile/VIP), country, wait_for. Not in any default chain.	`proxy=scrapedo&render_js=1&premium=1`
scamadviser_cf	Site-specific CF Puppeteer worker for scamadviser.com check-website pages.	`proxy=scamadviser_cf,scrapeowl`
trustpilot_cf	Site-specific CF Puppeteer worker for trustpilot.com review pages.	`proxy=trustpilot_cf,scrapeowl`
none	Direct fetch from our server (no proxy). Use for APIs where your server IP is whitelisted.	`proxy=none`

TODO / future cleanup: the site-specific scamadviser_cf and trustpilot_cf workers duplicate most of what browser_fetch now does. New integrations should use browser_fetch with the caller parsing the returned HTML. Existing callers stay on the dedicated workers until they migrate.

Timeouts: Total defaults to 30s (max 120s). Connect capped at 10s — fails fast to try next proxy. Browser-rendering proxies need timeout=45 or higher due to ~25s CF cold start.

Caching

Default TTL: 7 days. Cache key = URL + method + POST body.

Priority	Source	Description
1	cache=N	Per-request override (days)
2	Cache rules	Admin URL patterns (e.g. %facebook.com% = 30d)
3	Global default	7 days

Rate Limits

Limit	Default	Scope
Customer/hour	1,000	Per API key
Caller IP/hour	2,000	Per server IP
Originator IP/hour	100	Per end-user IP
Customer/day	Unlimited	Daily cap

Code Examples

Simple GET

cURL

curl -s 'https://www.botanon.com/api/fetch?url=https://example.com&apikey=YOUR_KEY&tag=test&ua=random' | jq .

PHP

$params = http_build_query(['url' => 'https://example.com', 'apikey' => 'YOUR_KEY', 'tag' => 'test', 'ua' => 'random']);
$data = json_decode(file_get_contents("https://www.botanon.com/api/fetch?{$params}"), true);
echo $data['body'];

Python

import requests
resp = requests.get("https://www.botanon.com/api/fetch", params={
    "url": "https://example.com", "apikey": "YOUR_KEY", "tag": "test", "ua": "random",
}, timeout=35)
data = resp.json()
print(data["body"])

Node.js

const params = new URLSearchParams({url: 'https://example.com', apikey: 'YOUR_KEY', tag: 'test', ua: 'random'});
const data = await (await fetch(`https://www.botanon.com/api/fetch?${params}`)).json();
console.log(data.body);

With Custom Headers (Bing API)

Python

import json
resp = requests.post("https://www.botanon.com/api/fetch", json={
    "url": "https://api.bing.microsoft.com/v7.0/search?q=test",
    "apikey": "YOUR_KEY",
    "headers": {"Ocp-Apim-Subscription-Key": "YOUR_BING_KEY"},
    "proxy": "none",
    "tag": "bing",
})
results = json.loads(resp.json()["body"])

JS Rendering + Geo-targeting

cURL

curl -s 'https://www.botanon.com/api/fetch?url=https://spa-app.com&apikey=YOUR_KEY&proxy=scrapeowl&render_js=1&premium=1&country=US&cache=30' | jq .

POST Data to Target

cURL

curl -s -X POST https://www.botanon.com/api/fetch \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://api.example.com/search","apikey":"YOUR_KEY","method":"POST","post_data":"{\"q\":\"test\"}","post_type":"json","tag":"search"}' | jq .

Compressed + Custom Cache TTL

cURL

curl -s --compressed 'https://www.botanon.com/api/fetch?url=https://example.com&apikey=YOUR_KEY&compress=1&cache=30' | jq .

Error Handling with Retry

Python

import requests, time

def fetch(url, tag="agent", retries=3):
    for i in range(retries):
        resp = requests.get("https://www.botanon.com/api/fetch", params={
            "url": url, "apikey": "YOUR_KEY", "tag": tag, "ua": "random",
            "refresh": 1 if i > 0 else 0,
        }, timeout=35)
        data = resp.json()
        if data["success"]:
            return data
        if data.get("error_code", "").startswith("RATE_LIMIT"):
            time.sleep(data.get("retry_after", 60))
            continue
        if data.get("http_code") in [404, 401, 403]:
            return data  # don't retry client errors
        time.sleep(2 ** i)
    raise Exception(f"Failed after {retries} attempts")

Google Maps API

Search Google Maps, look up places, and queue or instantly run crawls. All endpoints require an apikey.

Base path: /api/gmaps/

Method	Path	Description
GET	`/api/gmaps/search`	Search Google Maps
GET	`/api/gmaps/place/{id}`	Look up scraped place
POST	`/api/gmaps/crawl`	Queue or instant crawl
GET	`/api/gmaps/stats`	Usage statistics

Search Places

GET /api/gmaps/search?apikey=KEY&query=QUERY

Param	Required	Default	Description
apikey	yes	—	API key
query	yes	—	Search string (e.g. `"bars Derby"`)
depth	no	1	Pagination depth: 1≈20 results, 5≈40-50, 10+≈~120
tier	no	auto	Force: `gosom`, `residential`, `scrapeowl`
refresh	no	0	`1` to bypass 30-day cache

Example

curl "https://www.botanon.com/api/gmaps/search?apikey=KEY&query=bars+Derby&depth=3"

Response

{
  "query": "bars Derby",
  "depth": 3,
  "tier": "residential",
  "cached": false,
  "duration_seconds": 30.3,
  "places": [
    {
      "place_id": "0x4879...",
      "title": "Belong",
      "category": "Bar",
      "review_rating": 4.9,
      "review_count": 329,
      "phone": "+44 ...",
      "website": "https://...",
      "latitude": 52.922,
      "longitude": -1.477,
      "photos": ["https://..."],
      "reviews": [{ "author": "...", "rating": 5, "text": "..." }]
    }
  ]
}

Place Lookup

GET /api/gmaps/place/{place_id}?apikey=KEY

Returns a previously-scraped place by Google place_id. Use /search or /crawl first to populate.

Example

# Place ID in URL path
curl "https://www.botanon.com/api/gmaps/place/0x2fb4e9b346dee95b:0xd5811e26f15ce74d?apikey=KEY"

# Or as query parameter
curl "https://www.botanon.com/api/gmaps/place?apikey=KEY&place_id=0x2fb4e..."

Response

{
  "success": true,
  "cached": true,
  "place": {
    "place_id": "0x2fb4e9b346dee95b:0xd5811e26f15ce74d",
    "title": "Protego Pest Control",
    "category": "Pest control service",
    "phone": "+44 7897 314478",
    "review_rating": 5.0,
    "review_count": 42,
    "photos": ["https://..."],
    "reviews": [...]
  }
}

Crawl (Queue / Instant)

POST /api/gmaps/crawl (JSON body)

Two modes: queue (background processing, ~40 queries/hour) or instant (synchronous, returns results in 10-60s).

Param	Required	Default	Description
apikey	yes	—	API key
query	yes*	—	Google Maps search string. Not required if `place_id` is provided
place_id	no	—	Google Place ID — auto-derives query from place's location + category
mode	no	`queue`	`queue` or `instant`
priority	no	5	1-9 (1=highest). Queue mode only
location	no	auto	Location label (e.g. "Sevenoaks, Kent")
region	no	—	Region label (e.g. "Kent")
category	no	auto	Category (e.g. "pest_control")
depth	no	5	Pagination depth (1-20)
tier	no	auto	Force tier
refresh	no	0	Bypass cache (instant mode only)

Queue mode example

curl -X POST https://www.botanon.com/api/gmaps/crawl \
  -H 'Content-Type: application/json' \
  -d '{
    "apikey": "YOUR_KEY",
    "query": "pest_control in Sevenoaks, Kent, GB",
    "mode": "queue",
    "priority": 1,
    "region": "Kent",
    "category": "pest_control"
  }'

Queue response (201)

{
  "success": true,
  "action": "queued",
  "queue_id": 35361,
  "query": "pest_control in Sevenoaks, Kent, GB",
  "priority": 1,
  "message": "Queued at priority 1. Crawler processes ~40 queries/hour."
}

Instant mode example

curl -X POST https://www.botanon.com/api/gmaps/crawl \
  -H 'Content-Type: application/json' \
  -d '{
    "apikey": "YOUR_KEY",
    "query": "restaurants in Albufeira, PT",
    "mode": "instant",
    "depth": 3
  }'

Instant response (200)

{
  "success": true,
  "action": "instant",
  "query": "restaurants in Albufeira, PT",
  "tier": "residential",
  "duration_seconds": 28.5,
  "places_count": 12,
  "places": [ ... ]
}

Priority system: 1 = highest, 5 = normal (default), 9 = lowest. Crawler processes queue in order: priority ASC, then oldest first.

Duplicate detection: If the query already exists in the queue or history, the API returns the existing entry instead of creating a duplicate.

Crawl by place_id — crawl the area around a known place

curl -X POST https://www.botanon.com/api/gmaps/crawl \
  -H 'Content-Type: application/json' \
  -d '{
    "apikey": "YOUR_KEY",
    "place_id": "ChIJqfzVUcwEdkgR7cvsVYJ8daE"
  }'

place_id response (201) — note the place_context

{
  "success": true,
  "action": "queued",
  "queue_id": 35401,
  "query": "Pub in London, GB",
  "priority": 5,
  "location": "London",
  "category": "Pub",
  "place_context": {
    "source_place_id": "ChIJqfzVUcwEdkgR7cvsVYJ8daE",
    "source_title": "The Lamb & Flag, Covent Garden",
    "derived_location": "London",
    "derived_category": "Pub"
  }
}

place_id with category override — find restaurants near a pub

curl -X POST https://www.botanon.com/api/gmaps/crawl \
  -H 'Content-Type: application/json' \
  -d '{
    "apikey": "YOUR_KEY",
    "place_id": "ChIJqfzVUcwEdkgR7cvsVYJ8daE",
    "category": "restaurant",
    "priority": 2
  }'
# → Derives: "restaurant in London, GB"

place_id usage: The place must already exist in the DB (from a prior search/crawl). The API looks up its city + category and builds a crawl query for that entire area — it does not scrape just the single place. You can override the derived category with the category param.

Stats

GET /api/gmaps/stats?apikey=KEY

Returns your Google Maps API usage statistics.

Twitter/X Search API

Search Twitter/X using authenticated accounts with cookie-based auth. Results are cached for 24 hours.

Method	Path	Description
POST	`/api/twitter-search`	All actions via JSON body

Search Tweets

POST /api/twitter-search (JSON body)

Param	Required	Default	Description
apikey	yes	—	API key
action	no	`search`	`search`, `add-account`, or `accounts`
q	yes*	—	Search query (supports Twitter operators: `OR`, `from:`, `since:`, `-filter:`)
max	no	20	Max results (capped at 50)
no_cache	no	false	`true` to bypass 24h cache

Example

curl -X POST https://www.botanon.com/api/twitter-search \
  -H 'Content-Type: application/json' \
  -d '{
    "apikey": "YOUR_KEY",
    "q": "faro airport queue OR delay OR busy -filter:retweets",
    "max": 10
  }'

Response

{
  "success": true,
  "query": "faro airport queue OR delay OR busy",
  "cached": true,
  "cache_age_minutes": 45,
  "tweet_count": 15,
  "tweets": [
    {
      "id": "1234567890123456789",
      "text": "Massive queue at Faro airport...",
      "author": "traveller_uk",
      "author_name": "UK Traveller",
      "created_at": "2026-05-06T14:30:00Z",
      "retweet_count": 5,
      "like_count": 12,
      "url": "https://x.com/traveller_uk/status/1234567890123456789"
    }
  ]
}

Manage Accounts

Cookies are obtained from Chrome DevTools (Application → Cookies → x.com). They last ~1-2 months.

Add account

curl -X POST https://www.botanon.com/api/twitter-search \
  -H 'Content-Type: application/json' \
  -d '{
    "apikey": "YOUR_KEY",
    "action": "add-account",
    "username": "myxaccount",
    "auth_token": "abc123...",
    "ct0": "xyz789..."
  }'

List accounts

curl -X POST https://www.botanon.com/api/twitter-search \
  -H 'Content-Type: application/json' \
  -d '{"apikey": "YOUR_KEY", "action": "accounts"}'

Rate limiting: 6 accounts are loaded with round-robin rotation. ~4 searches per day per account. Results cached 24h. Use no_cache sparingly.

API Documentation

Contents

Endpoint

Authentication

Parameters

Required

Request Options

Proxy Options

Cache Options

Custom Headers

Response Format

Success (HTTP 200)

Failure (HTTP 502)

Rate Limited (HTTP 429)

Response Field Reference

Success Fields

Failure Fields

Error Codes

Proxy Chain

Special proxies (explicit only)

Caching

Rate Limits

Code Examples

Simple GET

With Custom Headers (Bing API)

JS Rendering + Geo-targeting

POST Data to Target

Compressed + Custom Cache TTL

Error Handling with Retry

Google Maps API

Search Places

Place Lookup

Crawl (Queue / Instant)

Stats

Twitter/X Search API

Search Tweets

Manage Accounts