BS
Scraping Frameworks
BeautifulSoup proxy integration
BeautifulSoup parses HTML but doesn't fetch it — you pair it with Requests (or HTTPX) to download pages. Route the fetch through Aethyn residential proxies, then parse the response with BeautifulSoup.
Setup steps
- 1Install both: pip install requests beautifulsoup4.
- 2Fetch the page through the Aethyn proxy with Requests.
- 3Parse response.text with BeautifulSoup.
- 4Add targeting suffixes for localized content.
Code
Requests + BeautifulSoup
import requests
from bs4 import BeautifulSoup
proxy = "http://aethyn-XXXXX-country-us:PASSWORD@proxy.aethyn.io:2099"
proxies = {"http": proxy, "https": proxy}
resp = requests.get("https://example.com", proxies=proxies, timeout=30)
soup = BeautifulSoup(resp.text, "html.parser")
print(soup.title.string)Replace aethyn-XXXXX and PASSWORD with your dashboard credentials. See the full targeting reference and protocol/port list.
Tips
- BeautifulSoup only parses — the proxy lives on the Requests/HTTPX call.
- Use lxml as the parser for speed on large pages.
- Rotate IPs per request for broad crawls; the default mode handles this.
BeautifulSoup proxy FAQ
Does BeautifulSoup need its own proxy config?
No. BeautifulSoup parses HTML you already fetched. Configure the proxy on the HTTP client (Requests or HTTPX) that downloads the page.
Docs, solutions & blog
Premium residential proxiesCost-efficient rotating residential pool from €2.00/GB for bulk scraping and monitoring.Elite residential proxiesCurated high-trust pool with city and ISP targeting for hard anti-bot targets.Quickstart docsCredentials, endpoints, and your first proxied request.Proxy glossaryPlain-English definitions for residential proxies, rotation, anti-bot, and scraping terms.