CR
Scraping Frameworks
Crawlee proxy integration
Crawlee (by Apify) is a Node.js crawling framework with built-in proxy rotation via ProxyConfiguration. Provide your Aethyn endpoint as a proxy URL and Crawlee routes its requests through residential IPs.
Setup steps
- 1Install Crawlee: npm install crawlee.
- 2Create a ProxyConfiguration with your Aethyn proxy URL.
- 3Pass it to your crawler (Cheerio/Playwright/Puppeteer).
- 4Add targeting suffixes in the username.
Code
Crawlee with ProxyConfiguration
import { CheerioCrawler, ProxyConfiguration } from "crawlee";
const proxyConfiguration = new ProxyConfiguration({
proxyUrls: ["http://aethyn-XXXXX-country-us:PASSWORD@proxy.aethyn.io:2099"],
});
const crawler = new CheerioCrawler({
proxyConfiguration,
async requestHandler({ $, request }) {
console.log(request.url, $("title").text());
},
});
await crawler.run(["https://example.com"]);Replace aethyn-XXXXX and PASSWORD with your dashboard credentials. See the full targeting reference and protocol/port list.
Tips
- Crawlee retries failed requests and rotates proxies automatically.
- Swap CheerioCrawler for PlaywrightCrawler when you need a real browser.
- Use sticky sessions for flows that must hold one IP.
Crawlee proxy FAQ
Does Crawlee rotate proxies automatically?
Yes. ProxyConfiguration rotates across the provided proxy URLs, and Aethyn rotates exits per request, so you get rotation at both layers.
Docs, solutions & blog
Related solution guideEnd-to-end workflow that pairs with this integration.Premium residential proxiesCost-efficient rotating residential pool from €2.00/GB for bulk scraping and monitoring.Elite residential proxiesCurated high-trust pool with city and ISP targeting for hard anti-bot targets.Quickstart docsCredentials, endpoints, and your first proxied request.Proxy glossaryPlain-English definitions for residential proxies, rotation, anti-bot, and scraping terms.