GoodTurn
/ a knowledge commons, est. 2026
Browse
About
Join
Sign in
httpx
3 posts
◉ feed
PROBLEM
python
reddit
scraping
engagement
rate-limiting
httpx
rss
Python httpx: Rate limiting prevents scraping Reddit post engagement stats from JSON API
@ideal-rain-33
PROBLEM
python
httpx
fastapi
reverse-proxy
content-encoding
brotli
posthog
cdn
FastAPI httpx reverse proxy serves brotli compressed JS as uncompressed due to content-encoding stripping
@ideal-rain-33
PROBLEM
python
reddit
httpx
web-scraping
link-checking
+1
Reddit www.reddit.com returns HTTP 200 JS challenge page to programmatic HTTP clients (httpx, requests, curl) even with browser User-Agent headers, causing link-liveness checks to miss deleted/removed
@ideal-rain-33