TravelElite tier

How to Scrape Airbnb Listings

Airbnb's price isn't a property of a listing — it's a function of dates, guests, and length of stay — and a single map search only ever returns a capped slice of an area. This guide covers collecting short-term rental data correctly: pricing for real date ranges, full-area coverage via map tiling, and responsible handling of host data.

SLWritten by Sophie Laurent·Reviewed by Sarah Chen·Updated September 4, 2026
12 min read
Editorial board contrasting a listing's fake single nightly price with a real quote for dates, guests, and stay length.

Key takeaways

  • There is no single 'price' — nightly rate depends on check-in/check-out, guests, and stay length, so always query with concrete dates.
  • A map search caps results (~a few hundred per area), so tile the bounding box into sub-areas to get full coverage.
  • Airbnb is a React SPA backed by an internal GraphQL API; render it or replay the API with the page's public API key.
  • Key on the listing room ID; it's stable while URLs and titles change.
  • Host names, photos, and reviews are personal data — collect listing facts, minimize the rest, and respect privacy law.

On this page

What goes wrong when this scraper fails

Short-term rental analysts, property managers, and market-research teams want Airbnb supply, pricing, and occupancy for a city or neighborhood. Two things trip up naive scrapers. First, Airbnb prices dynamically: the same listing shows different nightly rates depending on the dates, guest count, and length of stay you search, so a 'price' collected without dates is meaningless. Second, a map search returns only a capped set of results for the viewport, so a single query for 'Lisbon' silently misses most of the supply. On top of that, Airbnb is a heavily-defended SPA, so datacenter IPs get blocked fast.

Why this failure mode happens

Airbnb is a React single-page app that loads data from an internal GraphQL API (the same one the site uses), and it computes price server-side from your search parameters — dates, guests, stay length — plus host pricing rules. The map UI deliberately caps how many results it returns per viewport to keep responses small, so full coverage requires many narrower searches. And like any high-value travel marketplace, it runs strong bot detection that flags datacenter IPs and abnormal velocity.

Four-stage pipeline tiling a map bounding box and matching records on room ID.
One map search returns only a capped slice. Tile the box, then match records on room ID.

Challenges that make this hard to automate

  • Dynamic pricing that changes with dates, guests, and length of stay
  • A per-search result cap that hides most of an area's supply
  • A React/GraphQL architecture where data isn't in the static HTML
  • Aggressive bot detection on a high-value travel target
  • Host and review data that carry privacy obligations

Approaches that usually fail

  • Datacenter proxies — blocked quickly on a defended travel site
  • Scraping one map search per city — captures only the capped top slice
  • Collecting price without dates — returns a number no guest would be quoted
  • Manual checks — accurate but hopeless across a whole market's supply

When residential proxies fix this — and when they cannot

Residential IPs in the target market read as real travelers, get past the bot detection that blocks datacenter ranges, and return the localized currency and availability a local searcher sees. A large rotating pool lets you run the many tiled searches needed for full-area coverage — and the repeated date-range pricing lookups — without any single IP tripping velocity limits.

Browser network pane intercepting public StaysSearch JSON keyed on listing room ID.
Key on the listing room ID. Titles and URLs change. Skip host photos and reviews.

How Aethyn residential proxies help here

Airbnb is a high-security, geo-sensitive target that needs both reputation and the IP volume to run many searches. Aethyn provides both through the username.

  • Elite high-trust residential IPs to get past Airbnb's bot detection
  • Country/city targeting so currency and availability match the market
  • A large rotating pool for tiled map searches and date-range pricing lookups
  • Sticky sessions to keep a multi-request listing flow coherent
  • Per-byte metering so recurring market sweeps stay cost-predictable

How to implement this with residential proxies

  1. 1

    Always search with concrete dates and occupancy

    Because price depends on the stay, fix the parameters before collecting: check-in, check-out, and guests. Decide your sampling policy (e.g. a 2-night midweek stay 30 days out) and apply it consistently so prices are comparable across listings and over time.

    cURL
    curl -x "http://aethyn-XXXXX-country-pt-city-lisbon:PASSWORD@proxy.aethyn.io:5499" \
      "https://www.airbnb.com/s/Lisbon/homes?checkin=2026-08-10&checkout=2026-08-12&adults=2"

    Field note: Lock a fixed 'pricing scenario' per study (same lead time, nights, and guests). Mixing date ranges across runs makes a price 'change' that's really just a different stay length — the most common error in rental price tracking.

  2. 2

    Tile the map to beat the result cap

    A single area search returns only a few hundred listings regardless of true supply. To get full coverage, split the area's bounding box into a grid of smaller boxes and search each, then deduplicate. Subdivide any tile that still returns a full (capped) page, since that means it's hiding more.

    Python (tiling)
    def tiles(sw, ne, n):
        # sw/ne = (lat, lng) corners; split into n x n sub-boxes
        (s, w), (no, e) = sw, ne
        dlat, dlng = (no - s) / n, (e - w) / n
        for i in range(n):
            for j in range(n):
                yield ((s + i*dlat, w + j*dlng),
                       (s + (i+1)*dlat, w + (j+1)*dlng))
    
    def search_area(sw, ne, fetch, cap=270):
        seen = {}
        for t_sw, t_ne in tiles(sw, ne, 3):
            results = fetch(t_sw, t_ne)            # one residential request per tile
            for r in results:
                seen[r["room_id"]] = r
            # if a tile is capped, recurse into it for full coverage
        return list(seen.values())

    Field note: Deduplicate tiles on room ID — adjacent boxes overlap at the edges and the same listing appears in several. Without dedup your 'supply count' is inflated by double-counting boundary listings.

  3. 3

    Get structured data from the GraphQL API or a rendered page

    Airbnb's search and listing data come from its internal GraphQL API, which the page calls with a public API key embedded in the HTML. You can either drive a headless browser through the proxy and read the rendered results, or capture that key and call the API directly. Reading intercepted JSON is cleaner and survives UI changes better than DOM scraping.

    Python (Playwright + intercept)
    from playwright.sync_api import sync_playwright
    
    captured = []
    def on_response(resp):
        if "/api/v3/" in resp.url and "StaysSearch" in resp.url:
            try: captured.append(resp.json())
            except Exception: pass
    
    with sync_playwright() as p:
        b = p.chromium.launch(proxy={
            "server": "http://proxy.aethyn.io:5499",
            "username": "aethyn-XXXXX-country-pt-city-lisbon",
            "password": "PASSWORD",
        })
        page = b.new_page(locale="pt-PT")
        page.on("response", on_response)
        page.goto("https://www.airbnb.com/s/Lisbon/homes?checkin=2026-08-10&checkout=2026-08-12&adults=2",
                  wait_until="networkidle")
        b.close()

    Field note: Intercepting the StaysSearch response gives you the same structured listing objects the UI renders from — room id, price breakdown, rating, coordinates — without scraping a single CSS class. It's the most durable way to read Airbnb.

  4. 4

    Collect listing facts, minimize host/review data

    Store listing attributes (room id, type, capacity, amenities, price for your scenario, rating, review count, coordinates). Treat host names, photos, and individual reviews as personal data: collect only what your use case justifies, and prefer aggregates (rating, count) over review text.

    Field note: Capacity and room type are essential for normalization — comparing a studio's nightly rate to a 4-bedroom's is meaningless. Always segment price analysis by capacity and room type.

  5. 5

    Schedule market sweeps and track over time

    Run tiled searches on a cadence through rotating residential IPs, store (room_id, scenario, price, availability, timestamp), and derive supply, median price, and occupancy proxies per area. Re-price popular listings more often and back off on any challenge.

Best practices that keep scrapers reliable

  • Fix a consistent date/guest scenario before collecting prices
  • Tile the bounding box and recurse into capped tiles for full coverage
  • Dedupe on room ID across overlapping tiles
  • Intercept the GraphQL/StaysSearch JSON instead of scraping the DOM
  • Segment price analysis by capacity and room type
  • Minimize host/review personal data and respect privacy law

Common mistakes that burn proxy budget

  • Collecting a 'price' with no dates, so it matches no real quote
  • Running one search per city and missing most of the supply
  • Counting boundary listings twice by not deduping on room ID
  • Comparing nightly rates across different capacities
  • Hoarding host names and review text with no lawful basis
  • Hammering a defended travel site from datacenter IPs

Common questions about this scraper problem

Why does the same listing show different prices?
Airbnb prices dynamically based on the dates you search, the number of guests, and the length of stay, plus host pricing rules. There's no single price — always collect for a fixed, documented scenario (specific dates, nights, and guests) so the numbers are comparable.
Why am I only getting a few hundred listings for a city?
Airbnb caps results per map search. To capture an area's full supply you must tile its bounding box into smaller boxes, search each, and deduplicate — recursing into any tile that comes back capped because it's still hiding listings.
How do I extract Airbnb data reliably?
Airbnb is a React SPA backed by an internal GraphQL API. The most durable approach is to drive a headless browser through a residential proxy and intercept the StaysSearch JSON responses, which contain structured listing objects, rather than parsing churned HTML class names.
What should I use as the listing key?
The room ID. URLs and titles change, but the room ID is stable and is the right key for deduping across overlapping map tiles and for tracking a listing's price over time.
Is scraping Airbnb legal?
Collecting public listing data is common for market research, but Airbnb's Terms restrict automated access and listings contain personal data (hosts, reviews). Limit collection to public listing facts, minimize personal data, and consult counsel for your jurisdiction and use case.
Which tier should I use?
Elite. Airbnb is a high-value, well-defended travel target, so the higher-reputation residential pool plus city targeting (for correct currency and availability) gives the most reliable results.
How do I estimate occupancy?
You can't read bookings directly, but tracking each listing's availability calendar over repeated crawls lets you infer booked-vs-available nights as an occupancy proxy. Collect availability for your scenario consistently and diff over time.
How do I keep a recurring market sweep affordable?
Aethyn bills per byte, so cache listing detail that rarely changes, re-price only popular listings frequently, and tile efficiently (don't over-subdivide sparse areas) to control request volume.
Share this article

Solve this with Aethyn residential proxies

Route your first request through rotating residential IPs in minutes, with transparent per-byte pricing.

All solutions