Error Handling

This guide covers how to handle errors returned by the SurfacedBy API in a way that is safe under load and resilient to transient failures.

Classify errors first

When a request fails, classify the error before deciding whether to retry:

• 4xx other than 429 - the request is wrong. Do not retry. Fix the input, fix the auth, or surface the error to the caller.
• 429 - you exceeded a rate limit. Retry after the delay in the Retry-After header.
• 5xx - transient server-side problem. Retry with exponential backoff.

The problem document's type field is the right switch point for this logic; see Errors.

Respect Retry-After

A 429 response always includes a Retry-After header giving the number of seconds to wait. Never retry before then. Sending requests while you are over quota does not reset your counter; it only wastes your own request budget.

import time
import httpx
 
def call_with_retry(url, headers, max_attempts=5):
    for attempt in range(max_attempts):
        resp = httpx.get(url, headers=headers)
        if resp.status_code == 429:
            delay = int(resp.headers.get("Retry-After", "1"))
            time.sleep(delay)
            continue
        if 500 <= resp.status_code < 600:  # codegen-ignore: literal 500
            time.sleep(2 ** attempt)
            continue
        return resp
    resp.raise_for_status()

Exponential backoff for 5xx

Retry 5xx responses with exponential backoff, not a tight loop. A typical pattern is 1 second, 2, 4, 8, 16, capped at a reasonable maximum (for example 30 seconds). Add a small random jitter to avoid retry stampedes when many clients recover at the same time.

Stop retrying after a bounded number of attempts (5 is reasonable). Persistent 5xx errors indicate a real problem; retrying forever hides it.

Idempotency for writes

Write requests can fail ambiguously: you might not know whether the server processed the request before the network failed. Use the Idempotency-Key header on every write. Set it to a UUID you generate for the logical operation and retry with the same key.

curl -X POST https://api.surfacedby.com/api/v1/ext/domains \
  -H "X-API-Key: sk_live_..." \
  -H "Idempotency-Key: 9f8e2b1a-0c4d-4a7e-8b3f-5d1e2a3c6d7f" \
  -H "Content-Type: application/json" \
  -d '{"domain": "yourbrand.com"}'

If the original request succeeded, the retry returns the same response. If the original failed, the retry runs for real. See Idempotency for details.

Safe retries checklist

• Classify the error. Retry only 429 and 5xx.
• Honour Retry-After.
• Use exponential backoff with jitter for 5xx.
• Cap retry attempts.
• Use idempotency keys for writes.
• Log request_id from the problem document so you can correlate retries.