Which defense layers are scoring your request
Enforcer mode
Library default MONITOR scores without hard-blocking until mode/threshold changes. Fail-open on risk-API errors allows traffic with no score.
Cookie / token
_px3 risk token; _pxhd optional year-lived cookie; _pxvid/pxvid and pxcts carried by the client. __pxvid (two underscores) is Code Defender — different product.
Challenge UX
HUMAN Challenge press-and-hold; hold duration varies. Identify via px-captcha meta and challenge template markers, not unverified div ids.
ABR JSON
When Accept/Content-Type JSON gates fire, 403 application/json with appId/blockScript-style fields — proves an enforcer decision, not which action code.
Network identity
Residential ASN/geo helps the thin no-cookie risk path; cannot complete press-and-hold or mint _px3.
How to implement this with residential proxies
- 1
Identify via _px3 / _pxhd and px-captcha meta
Parse Set-Cookie for _px3 and _pxhd. On HTML bodies, search for meta name="description" content="px-captcha". Optional: window._pxAppId on challenge pages. Do not require a vendor response beacon — often none exists.
Pythonimport re def px_signals(headers, body: str) -> dict: sc = headers.get("set-cookie") or headers.get("Set-Cookie") or "" if isinstance(sc, list): sc = "\n".join(sc) return { "px3": "_px3=" in sc, "pxhd": "_pxhd=" in sc, "px_captcha_meta": bool( re.search(r'name=["\']description["\']\s+content=["\']px-captcha["\']', body or "", re.I) or re.search(r'content=["\']px-captcha["\']\s+name=["\']description["\']', body or "", re.I) ), "app_id_js": "_pxAppId" in (body or ""), } - 2
Interpret mode honestly
Library defaults are MONITOR + blockingScore 100. Remote config and onboarding routinely change them. Clean historical 200s do not prove the enforcer is off forever; a block/challenge page proves blocking is active for that path now.
- 3
Three-way curl before touching the challenge UI
Compare plain, Chrome-UA, and residential. Log which legs set _px3/_pxhd versus which return px-captcha HTML. If all three get the challenge shell, IP rotation will not click press-and-hold for you.
bashU=https://TARGET/ A='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36' P=http://aethyn-XXXXX-country-us:PASSWORD@proxy.aethyn.io:5499 for tag in plain ua res; do case $tag in plain) opts=() ;; ua) opts=(-A "$A") ;; res) opts=(-A "$A" -x "$P") ;; esac echo "== $tag ==" curl -sS -D - -o /tmp/px.$tag "$@" "${opts[@]}" "$U" | grep -iE 'HTTP/|set-cookie: _px|content-type' grep -i 'px-captcha|__pxAppId' /tmp/px.$tag | head -n 3 done - 4
Handle ABR JSON as an enforcer signal
A 403 with Content-Type application/json and fields like appId / blockScript means Bot Defender answered a JSON-preferring client. It does not tell you captcha vs hard-block by itself — do not ingest it as catalog data.
- 5
Choose remediation without solving CAPTCHAs here
Reputation-shaped: Elite residential + sticky identity. Challenge-UI-shaped: real browser context under your compliance policy. This guide stops at diagnosis — no press-and-hold automation instructions.
What goes wrong when this scraper fails
Scrapers hit “Access to this page has been denied,” a press-and-hold interstitial, or a JSON 403 with an appId and assume they need CAPTCHA workers. Others see weeks of clean 200s (monitor / fail-open) then sudden blocks and blame proxies. Correct handling starts from SDK-visible artifacts: cookies, meta markers, and mode semantics.
Why this failure mode happens
PerimeterX/HUMAN merges sensor telemetry with an origin or edge enforcer. Fresh deployments may score quietly under MONITOR defaults; production tenants flip to blocking. Challenge templates embed px-captcha in the description meta. Advanced Blocking Response emits JSON for JSON-preferring clients. None of that is fixed by rotating IPs alone when the UI challenge is already showing.
How Aethyn residential proxies help here
Hold ISP/country steady with Elite sticky sessions while a browser-backed flow establishes cookies. Use Premium rotation only after you confirm the failure is reputation-shaped rather than challenge-UI-shaped.
- Elite HTTP 5499 with -session-*-lifetime-* for browser continuity
- ISP targeting when retail tenants skew by carrier
- Premium 2099 for inexpensive header/cookie surveys
- SOCKS5 3499 for Playwright SOCKS configs
- No CAPTCHA-solving product surface
Common questions about this scraper problem
What is the best PerimeterX challenge HTML marker?
What do MONITOR mode and blockingScore 100 mean?
Does _px3 mean I passed?
Will residential proxies finish press-and-hold?
Which Aethyn ports?
Is scraping HUMAN-protected public pages allowed?
Best practices that keep scrapers reliable
- Prefer px-captcha meta over unofficial DOM ids
- Never replay _px3 under a different User-Agent
- Remember __pxvid ≠ _pxvid
- Three-way test before buying challenge labor
- Public data + Terms only
Common mistakes that burn proxy budget
- Expecting a permanent x-px response beacon
- Calling monitor-mode 200s permanent allow
- Misreading ABR as business JSON
- Assuming proxies complete Human Challenge
- Confusing Code Defender __pxvid cookies