What goes wrong when this scraper fails
A site translated and localized for many markets is only correct if it behaves correctly from each market — and a lot of localization logic keys off the visitor's location. From a single office IP you can't see that German users get USD pricing, that a locale redirect loops for Brazilian visitors, or that hreflang points to the wrong regional URL. Localization testing means exercising the site from real in-country IPs across the language, currency, formatting, and routing dimensions — ideally automatically in CI, not as a manual scramble before launch.
Why this failure mode happens
Sites localize using signals tied to location — IP geolocation, geo-redirects, and locale negotiation — so the rendered language, currency, formats, and even which URL you land on depend on where the request appears to originate. Defects in that logic are therefore invisible unless you test from the affected geography. Manual checks and VPNs cover a couple of countries at best, so regressions slip through precisely in the markets the team can't easily reach.
Challenges that make this hard to automate
- Localization behavior that only manifests from specific geographies
- Multiple dimensions to verify: language, currency, formats, routing
- hreflang and geo-redirect correctness across many markets
- Scaling beyond the one or two countries a VPN can cover
- Catching localization regressions before they ship
Approaches that usually fail
- Testing from the office IP — sees only one market's behavior
- A consumer VPN — a handful of countries, no automation, shared/flagged IPs
- Manual spot-checks by regional staff — slow and inconsistent
- Trusting translations without testing geo logic — misses routing/currency bugs
When residential proxies fix this — and when they cannot
Residential IPs in each target market let your tests originate from where real users are, so the site applies the same localization it would for them — correct (or incorrectly) language, currency, formats, and redirects. Because you can target many countries and cities programmatically, you can validate every market automatically instead of relying on whoever happens to be in-country.
How Aethyn residential proxies help here
Localization testing is geography-first and benefits from automation. Aethyn makes per-market runs a username change.
- Country and city targeting to test from each exact market
- Premium pool — cost-efficient for high-volume automated checks
- Real residential IPs so geo logic behaves as it does for users
- Sticky sessions for multi-step localized flows (e.g. locale selection)
- Per-byte billing so a full CI market matrix stays predictable
How to implement this with residential proxies
- 1
Define the market matrix and what to assert
List the markets (country + language) you support and the assertions per market: expected language strings, currency symbol/code, number/date formats, and the canonical/hreflang URL. This matrix is your localization test plan.
Python (market matrix)MARKETS = [ {"country": "de", "city": "berlin", "lang": "de", "currency": "EUR"}, {"country": "gb", "city": "london", "lang": "en", "currency": "GBP"}, {"country": "jp", "city": "tokyo", "lang": "ja", "currency": "JPY"}, ]Field note: Drive assertions from your own localization source of truth (the translation files and pricing config), not hand-typed expectations — that way the test plan updates automatically when you add a locale.
- 2
Load the site from each market's IP
Run the page from a residential IP in the market so geo-redirects and IP geolocation behave as they do for real users. Capture the final URL (after redirects), the rendered language, and the displayed prices for assertion.
Python (Playwright per market)from playwright.sync_api import sync_playwright def load(market, url): with sync_playwright() as p: b = p.chromium.launch(proxy={ "server": "http://proxy.aethyn.io:2099", "username": f"aethyn-XXXXX-country-{market['country']}-city-{market['city']}", "password": "PASSWORD"}) page = b.new_page(locale=f"{market['lang']}-{market['country'].upper()}") page.goto(url, wait_until="domcontentloaded") data = {"final_url": page.url, "html_lang": page.get_attribute("html", "lang"), "body": page.content()} b.close() return dataField note: Test IP geolocation and the Accept-Language header separately. A site might localize correctly by language header but mis-route by IP (or vice versa) — isolating them tells you which layer has the bug.
- 3
Assert language, currency, and formats
Check that the page's html lang matches, that expected translated strings are present (and English placeholders absent), that currency and number/date formats match the locale. These are the defects users feel first.
Python (assertions)def check(market, data): issues = [] if not (data["html_lang"] or "").startswith(market["lang"]): issues.append(f"html lang != {market['lang']}") if market["currency"] not in data["body"]: issues.append(f"missing currency {market['currency']}") # flag untranslated leakage: known English UI strings on a non-en locale if market["lang"] != "en" and "Add to cart" in data["body"]: issues.append("untranslated English string present") return issuesField note: Test for the absence of source-language strings, not just the presence of translations. Partial translations are the most common l10n bug, and they only show up when you assert that English UI text is gone on a non-English locale.
- 4
Verify hreflang and geo-redirects
Confirm hreflang tags point to the correct regional URLs, that the geo-redirect lands on the right locale for each country, and that there are no redirect loops or wrong-market landings. This routing layer is where silent, market-specific breakage hides.
Field note: Watch for redirect loops that only trigger from certain countries — a classic geo-redirect bug. Capture the redirect chain per market and assert it terminates on the expected locale URL.
- 5
Automate the matrix in CI
Run the market matrix on every release through residential IPs so localization regressions are caught before shipping, not reported by users. Fail the build on missing translations, wrong currency, or bad redirects, and store results to trend localization quality.
YAML (CI matrix)jobs: l10n: strategy: matrix: market: [de, gb, jp] runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: pip install playwright && playwright install chromium - run: python l10n_check.py --market ${{ matrix.market }} env: PROXY_USER: aethyn-XXXXX-country-${{ matrix.market }} PROXY_PASS: ${{ secrets.AETHYN_PASSWORD }} PROXY_HOST: proxy.aethyn.io:2099
Best practices that keep scrapers reliable
- Drive the test matrix from your localization source of truth
- Run each market from a real in-country residential IP
- Assert language, currency, and number/date formats per locale
- Test for absence of source-language strings, not just translations
- Verify hreflang and geo-redirects terminate on the right URL
- Automate the market matrix in CI and fail on regressions
Common mistakes that burn proxy budget
- Testing only from the office IP and seeing one market
- Relying on a VPN for a couple of countries with no automation
- Checking only that translations exist, missing partial leakage
- Ignoring geo-redirect loops that fire from specific countries
- Letting currency follow the IP without asserting the expected one
- Treating l10n as a pre-launch scramble instead of CI checks