Web Scraping CoreElite tier

Handle DataDome Blocks in Python (Same 403, Different Outcomes)

DataDome’s block page looks the same whether you hit a device check, a CAPTCHA, or a hard ban — HTTP 403, little branding, similar copy. The useful work is parsing the challenge payload and headers, then deciding whether a cleaner residential IP can help or whether your Python client fingerprint is the real problem.

JSWritten by Jordan Smith·Reviewed by Liam O'Connor·Last verified September 4, 2026
12 min read
Editorial board showing the same HTTP 403 with two outcomes: a DataDome-protected header versus a generic WAF.

Key takeaways

  • Trust x-datadome: protected and the datadome cookie — not the 403 status alone.
  • HTML and JSON 403 variants both happen; JSON with a single url key is easy to ingest by mistake.
  • Script host to expect: js.datadome.co/tags.js (not tags.datadome.co).
  • DataDome can fail open — intermittent 200s may mean degraded protection, not that your scraper improved.
  • Challenge query parameters can include t= values such as bv; treat them as hints, not oracle guarantees about IP quality.
  • Residential helps reputation and velocity; t-shaped client failures need browser-grade stacks.

On this page

Signal observed → what it means for your scraper

Match the signal you captured, then decide whether a residential proxy can change that layer before you rotate.

Signal observedWhat it indicatesWhich layerDoes a proxy change it
x-datadome: protectedCustom response header with literal value protectedDataDome integration active on the propertyConfirm datadome Set-Cookie and/or captcha-delivery URL; classify body before rotating.
403 HTML ~please enable JS…Nearly unbranded block/challenge HTMLDevice check, CAPTCHA, or ban sharing one envelopeParse embedded dd object or links to geo.captcha-delivery.com; do not branch on status alone.
403 JSON {"url": "https://geo.captcha-delivery.com/..."}resp.json() succeeds with one url keyAPI-shaped client Accept header received a challenge payloadReject single-key challenge JSON in validators; inspect t= in the query string as a hint only.
Set-Cookie: datadome=…Long-lived Secure cookie, typically not HttpOnlySession cookie minted for DataDome JS tag read/writeDo not forge it. Mirror browser cookie policy; pair with sticky residential if you continue in a real browser.
Intermittent 200 then 403Same client sometimes passesPossible fail-open / policy drift — not proof your code got smarterKeep content assertions on; do not celebrate flaky 200s as a durable fix.
403 unchanged across fresh residential IPsEvery exit gets the same challenge shellClient/TLS/JS layer dominates for this tenant pathMove to curl_cffi or Playwright; stop burning clean exits on a failing fingerprint.
Response-header detector rows for x-datadome protected, the datadome cookie, and a captcha body.
If x-datadome is missing, do not assume DataDome from a 403 alone.

How to diagnose and fix this scraper failure

  1. 1

    Detect DataDome from headers and cookies, not vibes

    Look for x-datadome: protected (literal value protected), Set-Cookie: datadome=…, and script references to js.datadome.co/tags.js. Optional headers such as x-datadome-cid may appear on challenges. Confirm before you assume Cloudflare or Akamai.

    Python (detect)
    def is_datadome(resp) -> bool:
        h = {k.lower(): v for k, v in resp.headers.items()}
        if h.get("x-datadome", "").lower() == "protected":
            return True
        sc = h.get("set-cookie", "")
        if "datadome=" in sc.lower():
            return True
        body = resp.text or ""
        return "js.datadome.co/tags.js" in body or "captcha-delivery.com" in body
  2. 2

    Classify HTML vs JSON 403 bodies safely

    Never treat a 403 JSON object with a single url key as business data. Extract the challenge URL and log query parameters for diagnosis. Values such as t=bv appear on some captcha URLs and correlate with hard IP pressure on that tenant — use them as hints, not as a guarantee that another IP will pass.

    Python (classify body)
    import json
    from urllib.parse import urlparse, parse_qs
    
    def classify_datadome_body(resp):
        text = (resp.text or "").strip()
        if text.startswith("{"):
            try:
                data = json.loads(text)
            except json.JSONDecodeError:
                data = None
            if isinstance(data, dict) and set(data) == {"url"}:
                q = parse_qs(urlparse(data["url"]).query)
                return {"kind": "json_challenge", "t": (q.get("t") or [None])[0], "url": data["url"]}
        if "captcha-delivery.com" in text or "var dd=" in text or "Please enable JS" in text:
            return {"kind": "html_challenge_or_block"}
        return {"kind": "other", "status": resp.status_code}

    Field note: Interstitial vs captcha field sets differ — do not assume every HTML 403 embeds a full dd object with a t field.

  3. 3

    Three-way curl before you rewrite the scraper

    Plain curl, Chrome-UA curl, and residential-proxied curl separate UA triggers from IP reputation. Record x-datadome and body class for each.

    bash
    PROXY="http://aethyn-XXXXX-country-fr:PASSWORD@proxy.aethyn.io:5499"
    URL="https://TARGET/"
    
    for label in plain ua res; do
      case $label in
        plain) extra=();;
        ua) extra=(-A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36");;
        res) extra=(-x "$PROXY" -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36");;
      esac
      echo "=== $label ==="
      curl -sS -D - -o "/tmp/dd-$label.body" "$URL" "${extra[@]}" -w "\nHTTP %{http_code}\n" | grep -iE "HTTP/|x-datadome|set-cookie: datadome|server:"
    done
  4. 4

    Branch: rotate residential only for IP-shaped pressure

    When fresh residential exits change the outcome, keep rotation or sticky sessions in Python via the proxy URL. When they do not, switch to curl_cffi or Playwright through the same proxy instead of expanding concurrency.

    Python (requests + proxy)
    import requests
    
    PROXY = "http://aethyn-XXXXX-country-fr:PASSWORD@proxy.aethyn.io:5499"
    proxies = {"http": PROXY, "https": PROXY}
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36",
        "Accept": "text/html,application/xhtml+xml",
    }
    r = requests.get("https://TARGET/", headers=headers, proxies=proxies, timeout=45)
    print(r.status_code, r.headers.get("x-datadome"), is_datadome(r))
  5. 5

    Assert success on content — DataDome can fail open

    Intermittent 200 responses can mean protection degraded or a custom allow path, not that your scraper is correct. Require expected selectors and reject challenge titles before writing rows.

What goes wrong when this scraper fails

Python scrapers talking to DataDome-protected classifieds, travel, and luxury retail properties often see a wall of HTTP 403s with almost no vendor logo. Because device check, CAPTCHA, and ban share that envelope, teams either rotate proxies forever or parse the JSON challenge as if it were API data. Correct handling starts by identifying DataDome explicitly, then reading the body.

When residential proxies fix this — and when they cannot

Residential proxies help when the failure is reputation or velocity shaped — including cases where challenge URLs show t=bv on a given tenant. They also spread request rate so you trip fewer rate/timebox policies. They do not run the DataDome JS tag or fix a Python TLS fingerprint that the edge dislikes. If every clean exit gets the same challenge shell, stop rotating and fix the client.

Form cards separating device-shaped DataDome checks from ASN-shaped blocks that a residential pool can address.
A device check still needs a browser. A pool change will not click the slider.

How Aethyn residential proxies help here

Use Aethyn to control the IP variable deliberately while your Python code classifies DataDome responses. Sticky Elite sessions support browser-backed flows; rotating Premium covers volume on softer paths.

  • Elite HTTP 5499 for higher-trust exits on hard tenants
  • Per-request rotation on Premium 2099 for velocity-sensitive catalogs
  • Country targeting for geo-differentiated properties
  • Documented username grammar for reproducible experiments
  • Honest positioning: proxy ≠ DataDome bypass

Best practices that keep scrapers reliable

  • Detect x-datadome before generic 403 handlers
  • Reject single-key captcha-delivery JSON as data
  • Run three-way diagnostics on each new tenant
  • Prefer sticky Elite exits for browser-backed sessions
  • Stay in public-data / ToS-safe scope — no CAPTCHA solve recipes

Common mistakes that burn proxy budget

  • Ingesting challenge JSON as API payloads
  • Forging or replaying datadome cookies across IPs
  • Assuming tags.datadome.co is the script host
  • Celebrating flaky 200s without content asserts
  • Rotating forever on a pure client fingerprint failure

Common questions about this scraper problem

Why does DataDome return HTTP 403 for every outcome?
Device check, CAPTCHA, and ban flows commonly share a 403 envelope with little branding. Distinguish them by headers (x-datadome), cookies (datadome), and body/challenge URL contents — not by status code alone.
How do I detect DataDome in Python?
Check for header x-datadome: protected, Set-Cookie names containing datadome=, script URLs on js.datadome.co/tags.js, or captcha-delivery.com challenge links.
Will a residential proxy remove DataDome CAPTCHAs?
No. Proxies change IP reputation and velocity. CAPTCHA and device-check clearance require a capable client environment. This guide does not cover solving CAPTCHAs.
What does t=bv mean on a captcha-delivery URL?
It is an observed challenge query parameter that often correlates with strong IP pressure on that tenant. Treat it as a diagnostic hint: try a clean residential exit, but do not treat any t= value as a mathematical proof about IP quality.
Why do some requests succeed without code changes?
DataDome can fail open or apply tenant-specific allow paths. Intermittent success is not a durable strategy — keep content assertions and classify failures explicitly.
Which Aethyn port should I use from Python?
Premium HTTP 2099 or Elite HTTP 5499 (SOCKS5 1099 / 3499 if your client needs SOCKS). Example: http://aethyn-XXXXX-country-us:PASSWORD@proxy.aethyn.io:5499.
Is it OK to scrape DataDome-protected sites?
Only collect public data you are permitted to access, follow robots.txt and Terms, and avoid account takeover or CAPTCHA abuse. Use this page for diagnosis of lawful collection pipelines.

References

Share this article

Solve this with Aethyn residential proxies

Route your first request through rotating residential IPs in minutes, with transparent per-byte pricing.

All solutions