curl_cffi vs Requests vs Playwright for Web Scraping (2026)

Use the least expensive client that reliably produces the required document. That is the whole job. Python requests is cheapest. curl_cffi costs a bit more and can make the transport resemble Chrome. Playwright is a real browser and the most expensive per page. Move up only when the cheaper client cannot see the document. Move down after a JavaScript page has already failed — that is cheaper than guessing.

A three-client scraping test holds the residential proxy username constant and changes only the HTTP client. You fetch the same URL three times: once with requests, once with curl_cffi and impersonate="chrome", once with Playwright. You record HTTP status and a body class. You do not publish a pass rate from this page. The residential exit is the experimental control, not a fourth client.

Anti-bot systems can evaluate signals beyond the IP address, including TLS characteristics, HTTP/2 behavior, JavaScript execution, cookies, and browser/session consistency. Buying more residential bandwidth does not execute JavaScript. Changing the User-Agent does not make requests use Chrome's TLS stack or reproduce Chrome's TLS fingerprint. Launching Chromium on a datacenter ASN does not make the handshake look like a household.

The table is a starting map. The class you actually get still depends on the destination — a JSON API, a server-rendered article, a hydrated offer grid, a consent wall, or a challenge page.

Copy-paste for curl_cffi lives at the curl_cffi integration. Launch versus context, and credentials in fields, live in Playwright with residential proxies. What JA3 and JA4 actually measure lives in TLS fingerprinting for scrapers. This page does not duplicate those three jobs.

When to use each client

Use requests when the payload is an API or JSON endpoint, simple server-rendered HTML, and there is no meaningful TLS or browser challenge.

Use curl_cffi when HTTP is sufficient, TLS characteristics matter, JavaScript is not required, and you want lower overhead than a browser.

Use Playwright when JavaScript renders the target content, browser cookies or storage matter, DOM interaction is required, or the target requires browser execution.

What each client actually is

Python requests is an HTTP library on OpenSSL (or whatever TLS stack your environment ships). It can send a Chrome User-Agent. Changing that header does not make it use Chrome's TLS stack or reproduce Chrome's TLS fingerprint. Against a destination that only checks the IP, a residential exit is often enough. Against a destination that scores library handshakes, it is still a named library. The encyclopedia for that handshake is TLS fingerprinting, not this page.

curl_cffi is a Python binding for curl-impersonate. impersonate="chrome" is a rolling alias for a current Chrome profile (the library default moves; do not freeze chrome124 in new copy). It aligns cipher order, extensions, and HTTP/2 frames with that profile, and injects matching default headers unless you set default_headers=False. That is the TLS and HTTP/2 layer. It is not a browser. It does not run JavaScript, does not paint a DOM, and does not clear a JS challenge. Chrome TLS impersonation can make the transport resemble a browser, but it does not reproduce browser execution, DOM behavior, Web APIs, or JavaScript-generated state.

Playwright is a real Chromium (or Firefox, or WebKit) whose every request — document, XHR, WebSocket — can exit through the same residential username. It executes scripts. It is also the most expensive client per page, and the easiest one to misconfigure: Chromium strips user:pass from a proxy URL and answers 407. Credentials go in username and password fields. Per-context proxies on Chromium need launch({ proxy: { server: "per-context" } }).

The residential exit is a fourth thing, not a fifth client. It rewrites the address and ASN. It does not pick the HTTP library, run JavaScript, or hide a library handshake. How one vendor encodes country, city, and sticky lifetime is later — How Aethyn fits the test.

Three-client test (run this, then cite your classes)

Hold these constants: URL, language, device class, proxy username, sticky session id, and collection window. Change only the client.

1. Pin the exit. Hold the same proxy session for the three fetches. Verify with an IP lookup through that session before the first target request. Record exit_ip, resolved_city, resolved_isp.
2. Fetch with requests. Same proxy URL, timeout 30. Store http_status, body length, and a class from the list below.
3. Fetch with curl_cffi. Same proxy URL, impersonate="chrome". Store the same columns. Do not reuse the requests cookie jar.
4. Fetch with Playwright. Same username and password in fields, not in the server string. New context, not a reused one. Store the same columns after the document is idle enough to classify.
5. Classify the body before you insert a row. Status 200 with a challenge page is not a pass.

Body classes (fail-closed):

Do not average these into a “success rate.” A challenge on requests and a populated on curl_cffi is a TLS story. A challenge on both HTTP clients and a populated grid in Playwright is a JavaScript story. A challenge on all three is not a client bug — classify the wall (Cloudflare, DataDome, consent) before you buy more IPs.

Fictional example (not a measurement). Suppose requests returns 200 / challenge, curl_cffi returns 200 / challenge, Playwright returns 200 / populated. The layer that moved is JavaScript, not the handshake and not the ASN. Do not cite those three outcomes as Aethyn data. Replace them with your run and your collected_at.

Public web only. Respect robots.txt, rate limits, and Terms. This protocol samples publicly rendered pages. It is not a login, checkout, CAPTCHA, or Cloudflare-bypass recipe.

Decision table

Use the least expensive client that reliably produces the required document. Moving up costs CPU, bandwidth, and operational complexity. Moving down after a JS page has already failed is cheaper than guessing.

When the destination is Cloudflare-shaped and the document never appears without a JS challenge, the collector playbook is scrape behind Cloudflare — still not a bypass. When the page loaded and the HTML is still a deny, that is Playwright silent 403.

How Aethyn fits the test

A browser download is larger than an HTTP GET. That is not a reason to avoid Playwright when the grid is JavaScript. It is a reason not to use Playwright for a JSON API. The thesis still holds on this network: pick the cheapest client that returns the document, then encode location and sticky on the username.

Identity has two layers and they are easy to glue together wrongly:

• Session() / Playwright context is the cookie jar and storage.
• Aethyn -session-<id>-lifetime-N is the exit IP on a rotating pool (1–1440 minutes, default 30).

A curl_cffi Session() on a rotating username will send cookies from identity A out of IP B. A Playwright context reused across session ids does the same. Hold sticky for the flow that minted the cookie; start a new session id for the next identity. Sticky is not a dedicated IP. When the window ends, the pool may hand you a different household in the same country — verify again if the job continues.

Premium versus Elite is location, not “stealth.” Country catalog and currency: Premium HTTP 2099 / SOCKS5 1099. City or ISP: Elite HTTP 5499 / SOCKS5 3499. A city suffix on Premium is refused, not silently ignored. Asking for a city is not landing there — if resolved_city does not match, do not label the row as that city. SOCKS5 does not hide TLS; it tunnels TCP. Copy-paste usernames are on the curl_cffi integration and the targeting docs.

Soft 200s

HTTP 200 with unusual-traffic, a consent wall, or an empty app shell will poison any comparison you later call a pass rate. Classify before you store. The longer treatment is HTTP 200 access denied.

curl_cffi can return 200 with Chrome-shaped TLS and still serve a challenge HTML page. Playwright can return 200 with a rendered deny. The status code is a transport fact. The body class is the measurement.

What this page is not

It is not a JA3 encyclopedia — TLS fingerprinting already is. It is not a Playwright setup guide — Playwright residential proxies already is. It is not a curl_cffi copy-paste card — the integration already is. It is not a claim that curl_cffi “bypasses Cloudflare,” clears CAPTCHAs, or beats Playwright on a published percentage.

If you need a number for a slide, run the three clients, store the schema, and put your collected_at on the slide.