What goes wrong when this scraper fails
An advertiser is paying for premium placements in eight countries and needs to confirm three things: the right creative is serving, it's appearing in brand-safe placements, and the landing pages are compliant. An ad-ops team separately needs to catch fraud — misdeclared geos, domain spoofing, and cloaked offers. From a single office IP you see exactly one version of reality, and it's the version served to your home country from a corporate network — not what a real consumer in your target markets experiences.
Why this failure mode happens
Ad servers, DSPs, and affiliate networks target by geography, device, and audience, so the creative and landing page genuinely differ by location. Worse, bad actors actively cloak: their systems fingerprint the visitor and serve a benign page to datacenter IPs, known crawler ranges, and anything that looks like verification — while serving the real, often non-compliant, page to ordinary residential users. Verifying from a datacenter IP doesn't just miss geo variation; it can be deliberately shown a fake.
Challenges that make this hard to automate
- Geo-targeted creative and offers that differ by country and city
- Cloaking that detects verification traffic and hides the real landing page
- Multi-hop affiliate/redirect chains that obscure the final destination
- Detecting misplacement, domain spoofing, and geo-masking fraud across regions
- Datacenter IPs that ad platforms discount or serve differently than real users
Approaches that usually fail
- VPNs — few locations, shared IPs, and easily fingerprinted as non-residential
- Asking colleagues abroad to check manually — slow, inconsistent, and unauditable
- Datacenter proxies — discounted or actively cloaked by the systems you're checking
- Single-region monitoring — structurally blind to the geo-specific issues that matter most
When residential proxies fix this — and when they cannot
Residential proxies make verification traffic originate from real consumer IPs in each target market, so you observe the exact creative, placement, and landing experience local users get — including the real page that cloaking tries to hide from non-residential visitors. That combination of correct geography and genuine-user appearance is what makes ad verification and geo-fraud detection actually trustworthy.
How Aethyn residential proxies help here
Verification needs broad geographic reach and IPs that ad systems treat as real users rather than discounting. That's the Premium tier's profile, with Elite available where city precision or a tough target demands it.
- 195+ country coverage to verify campaigns in every market they run
- Country targeting (Elite city) for region-accurate creative and offers
- Residential IPs that ad systems and cloakers treat as genuine users
- Sticky sessions to resolve an entire redirect chain from one consistent IP
- Premium pricing that makes broad, repeated verification coverage affordable
How to implement this with residential proxies
- 1
Define the campaign matrix you're verifying against
Verification is comparison, so start with the expected state: which creative should serve, in which placements, on which publisher URLs, in which countries (and cities on Elite). Without this ground truth you're just collecting screenshots; with it, every observation either confirms the plan or flags a deviation.
Field note: Include device and OS expectations in the matrix where campaigns target them. A creative that looks fine on desktop but broken on mobile is a real delivery defect that desktop-only verification will miss.
- 2
Render publisher pages from each target market
Drive a real browser through a residential IP in the target country — ads render client-side, so raw HTTP won't see them. Capture which creatives and placements appear, and screenshot the rendered page as tamper-evident evidence.
Python (Playwright)from playwright.sync_api import sync_playwright def check(country, url): with sync_playwright() as p: b = p.chromium.launch(proxy={ "server": "http://proxy.aethyn.io:2099", "username": f"aethyn-XXXXX-country-{country}", "password": "PASSWORD", }) page = b.new_page(locale="en-GB") page.goto(url, wait_until="networkidle") ads = page.query_selector_all("iframe[id*='google_ads'], [id*='ad'], [class*='ad-']") page.screenshot(path=f"evidence/{country}.png", full_page=True) b.close() return len(ads) print(check("gb", "https://publisher.example/article"))Field note: Match the browser locale and timezone to the proxy country. A GB IP rendering an en-US, America/New_York browser is an obvious mismatch that some ad systems use to decide you aren't really a local user — and to serve you the cloaked creative.
- 3
Follow the full redirect chain on one IP
Affiliate and cloaked offers hide behind chains of redirects. Pin a sticky session so every hop resolves from the same residential IP, and record each Location header plus the final URL. A chain that lands somewhere other than the declared destination is the signature of cloaking.
Python (trace redirects)import requests def trace(country, click_url): proxy = f"http://aethyn-XXXXX-country-{country}-session-adcheck1-lifetime-10:PASSWORD@proxy.aethyn.io:2099" proxies = {"http": proxy, "https": proxy} r = requests.get(click_url, proxies=proxies, allow_redirects=True, timeout=30) hops = [resp.headers.get("location") for resp in r.history] return {"hops": hops, "final_url": r.url, "status": r.status_code} print(trace("gb", "https://adserver.example/click?id=123"))Field note: Compare the final URL against the campaign matrix's expected domain. Domain spoofing and cloaking show up here first: the click that's supposed to reach brand.com quietly terminates on a look-alike or a non-compliant offer.
- 4
Record evidence and flag deviations
Store the screenshot, the redirect chain, the final URL, and the detected creatives per market with a timestamp, then diff against the expected matrix. Surface mismatches — wrong creative, missing placement, unexpected final domain, geo where it shouldn't serve — as actionable alerts rather than raw logs.
- 5
Re-verify on a schedule
Creative rotates, flights change, and fraudsters turn cloaking on after the campaign launches to evade early checks. Re-run verification across markets on a cadence and keep the history, so you catch drift and compliance issues while there's still budget to protect.
Best practices that keep scrapers reliable
- Verify against an explicit campaign matrix, not just by collecting screenshots
- Use a residential IP in each exact target market, with matching locale and timezone
- Capture screenshot + full redirect chain + final URL as timestamped evidence
- Resolve the whole redirect chain on one sticky IP
- Diff observations against the expected plan and alert on deviations
- Re-verify on a schedule, since creative rotates and cloaking activates late
Common mistakes that burn proxy budget
- Verifying from one country and assuming global parity
- Using datacenter IPs that get the clean, cloaked version of the page
- Mismatching IP country with browser locale/timezone and tipping off cloakers
- Rotating mid-redirect and altering or breaking the landing chain
- Storing raw logs with no comparison to the expected campaign matrix
- Checking once at launch instead of on an ongoing schedule