How to Collect Public Reddit Posts

What is in scope vs out of scope

What counts as a public Reddit post?

Collecting public Reddit posts means reading listings and comments a logged-out visitor can already see. The official path is Reddit's Data API: you register an app, request a token, and call oauth.reddit.com with a bearer header and a descriptive User-Agent. A residential proxy does not replace that token. It is the exit the token's requests leave from, so a run of public reads is not one cloud address at Reddit's edge. This page stays on that job. A step that needs a browser session cookie, a vote, a comment, or another person's account is out of scope. Reddit changes the allowance on the Data API, so check the current docs before you size a job. This guide does not publish a rate number. The product page for the exit itself is the Reddit proxies guide. This page is the read path.

Why a cloud address fails a public read

Reddit scores the address a request comes from, including on endpoints that do not require a user. A burst from one datacenter IP collects 429s and challenge pages while the same public listing still opens in a normal browser. A public .json URL on www.reddit.com looks like a shortcut, and it is only a fallback for a page a logged-out browser can open. If that URL returns a sign-in wall or a challenge, stop and use the official API. Do not treat the fallback as a bypass. A thread is one conversation: pagination and a follow-up comment belong on the exit that fetched the first page. A new thread is a new job. Reusing the session id inside its window can return the same exit, which is the right retry for that thread and the wrong identity for the next one.

Read a public listing without an account action

1. 1

   Register your own app

   Create an app in your own Reddit account and copy the client id and client secret. The token you request is for that app. If the app type asks for the owner's username on the token endpoint, that password is yours and it goes only to Reddit's token URL. It is not a browser login, and it is not used for any other account.

   Field note: Send a specific User-Agent. Reddit rejects generic clients.

2. 2

   Request a token, then read a public listing

   POST grant_type=client_credentials to https://www.reddit.com/api/v1/access_token with HTTP Basic auth of client id and client secret. Call https://oauth.reddit.com/r/{subreddit}/hot with the bearer token. If that grant is refused for your app type, follow Reddit's current token example for the app you created, still with only the owner's credentials. Do not fall through to a login form.

   pythonCopy

   import requests
   
   proxy = "http://aethyn-XXXXX-country-us-session-reddit1-lifetime-30:PASSWORD@proxy.aethyn.io:2099"
   proxies = {"http": proxy, "https": proxy}
   ua = "public-listings/0.1 (by your-app-name)"
   
   token = requests.post(
       "https://www.reddit.com/api/v1/access_token",
       auth=("CLIENT_ID", "CLIENT_SECRET"),
       data={"grant_type": "client_credentials"},
       headers={"User-Agent": ua},
       proxies=proxies,
       timeout=30,
   )
   token.raise_for_status()
   access = token.json()["access_token"]
   
   listing = requests.get(
       "https://oauth.reddit.com/r/python/hot",
       headers={"Authorization": f"bearer {access}", "User-Agent": ua},
       params={"limit": 25},
       proxies=proxies,
       timeout=30,
   )
   print(listing.status_code, listing.json()["data"]["children"][0]["data"]["title"])

3. 3

   Verify the exit before the real listing

   Call an IP-lookup URL through the same proxy before oauth.reddit.com. The country in that response should match -country- on the username. If it does not, close the job and open another with a new session id. Do not continue on a lookup that already disagrees with the job.

4. 4

   Classify the body, then start the next thread clean

   Store a row only when the JSON contains posts. An empty data list, a sign-in wall, or a challenge page is a failed read. A public .json URL is allowed only as a fallback for a page a logged-out browser can open, and only after the official call has been set up. When the thread is done, leave the session. The next thread uses a new session id.

What the proxy changes, and what it does not

The proxy changes the household the official request leaves from. It does not grant a token, raise Reddit's allowance, or make an account action allowed. Premium HTTP on port 2099 carries country targeting in the username. Elite HTTP on port 5499 adds a supported city or ISP when you need that finer exit, and it does not guarantee Reddit will accept the session. Omit -session- when each independent listing can rotate. Add -session-ID-lifetime-N, from 1 to 1440 minutes and default 30, only while one thread must stay on one exit.

How Aethyn residential proxies help here

Point the HTTP client at proxy.aethyn.io. Keep the Reddit token in the Authorization header. Keep the proxy password in the proxy password, not in the Reddit request.

• Premium HTTP on port 2099 for country-level public reads
• Elite HTTP on port 5499 when a supported city or ISP is required
• Username tokens -country-, -session-, and -lifetime- in that order
• Sticky 1–1440 minutes, default 30, on a rotating-pool IP
• The same pool on SOCKS5 ports 1099 and 3499 if the client speaks SOCKS5

Public Reddit collection questions

Best practices that keep scrapers reliable

• Keep the Reddit bearer token and the proxy password in different fields.
• One session id per thread. A new id when the next thread should leave from somewhere else.
• Read Reddit's current Data API page before you raise volume.
• Link the Reddit proxies page for the product choice, and keep this page on the read path.

Common mistakes that burn proxy budget

• Putting a SOCKS URL into an HTTP client that expects http://user:pass@host:port.
• Reusing session-reddit1 after the lifetime ends and assuming the IP is still pinned.
• Saving a 200 challenge document as posts.
• Using someone else's account, or a browser cookie, because the token call was inconvenient.