Playwright Silent 403: Your Scraper Was Blocked Quietly

The anti-bot page loaded. Playwright shrugged. page.goto handed back a Response with .status 403 and raised nothing. Your next exception was a timeout waiting for #product-price — minutes of confusion later. Selenium’s driver.get is quieter still: no exception, driver.title becomes Access Denied.

This is the Class-3 error-string problem: the library never framed the event as a block. Full pipeline: Your scraper framework did not tell you it was blocked.

Why engineers search the wrong string

Incident chats fill with Timeout 2500ms exceeded / waiting for locator("#product-price") / empty Selenium TimeoutException messages. Those strings describe the second failure. The first failure was a deny document that completed navigation successfully. Until you log response.status and page.title() immediately after goto, every anti-bot vendor looks like a flaky selector.

That is why this post exists beside the vendor guides: Akamai, DataDome, and Kasada each have distinct receipts — but none of them matter if your framework never surfaces the HTTP event.

Verified behaviour (automation stacks)

Soft sibling: HTTP 200 Access Denied bodies also look “successful” to transport-level checks.

Scrapy’s default handling deserves a callout: many spiders log Ignoring response <403 …>: HTTP status code is not handled or not allowed and produce zero items with no exception in your parse callback. Enable explicit errbacks or handle 403 in the spider — silence is not success there either.

Classify before you wait

Code Snippet
from playwright.sync_api import sync_playwright

def goto_classified(page, url: str):
    resp = page.goto(url, wait_until="domcontentloaded", timeout=45_000)
    status = resp.status if resp else None
    title = page.title()
    if status is not None and status >= 400:
        raise RuntimeError(f"blocked_or_error status={status} title={title!r}")
    if "access denied" in (title or "").lower():
        raise RuntimeError(f"blocked_document title={title!r}")
    return {"status": status, "title": title, "url": page.url}

Only after classification says the document is real should you call wait_for_selector on commerce nodes. Selector timeouts are symptoms, not diagnoses.

Selenium sketch of the same idea:

Code Snippet
from selenium import webdriver

def get_classified(driver, url: str):
    driver.get(url)  # raises nothing on HTTP 403 HTML
    title = driver.title or ""
    if "access denied" in title.lower() or "just a moment" in title.lower():
        raise RuntimeError(f"blocked_document title={title!r}")
    return title

Proxy 407 hang ≠ site block

Misconfigured Playwright proxy auth against a 407 can hang until the full goto timeout with no 407 string — observed as a long TimeoutError. A dead proxy typically fails fast with net::ERR_PROXY_CONNECTION_FAILED. Configure username/password first:

• Premium HTTP 2099 / SOCKS5 1099
• Elite HTTP 5499 / SOCKS5 3499

Example proxy server for Playwright: http://aethyn-XXXXX-country-us-session-pw1-lifetime-30:PASSWORD@proxy.aethyn.io:2099 — see Playwright integration. Prefer Elite HTTP 5499 once attribution confirms Bot Management–class targets; keep Premium for routine A/B after classification.

Three-way curl outside the browser

When the browser is silent, confirm the wire:

Code Snippet
URL=https://TARGET/
UA='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36'
curl -sS -o /dev/null -w 'plain %{http_code}\n' "$URL"
curl -sS -o /dev/null -w 'ua %{http_code}\n' -A "$UA" "$URL"
curl -sS -o /dev/null -w 'res %{http_code}\n' -A "$UA" \
  -x 'http://aethyn-XXXXX-country-us:PASSWORD@proxy.aethyn.io:2099' "$URL"

Then attribute with Identify which anti-bot blocked you — Akamai Reference strings, x-datadome: protected, Kasada UUID paths, and friends. Do not invent a bypass from a TimeoutError message.

Pipeline stages (memorize this order)

1. Navigate — capture Response/status (Playwright) or title/URL (Selenium); never assume throw-on-deny
2. Classify — block vs real document vs network/proxy failure
3. Assert — commerce selectors only after classification says the HTML is real
4. Attribute — send status, cookies, body prefix to the anti-bot hub
5. Remediate — one change: IP tier or client stack — not both at once in a panic

Blind except Exception: retry loops violate stage 5 and burn Elite exits on fingerprint failures.

Remediate once

Public-data scope only — this page does not teach CAPTCHA solving. Pair with sibling error posts when the classified document names a vendor: DataDome 403, Akamai Reference #18, Kasada 429.

Continue with the navigate → classify → assert → attribute → remediate stages: Framework silent blocks.