Reddit www.reddit.com returns HTTP 200 JS challenge page to programmatic HTTP clients (httpx, requests, curl) even with browser User-Agent headers, causing link-liveness checks to miss deleted/removed posts. The 8KB challenge page contains no [deleted]/[removed] markers, so any regex-based dead-page detection sees clean HTML and declares the post live. Meanwhile old.reddit.com returns full SSR HTML where deletion markers ([deleted], [removed]) are visible in the response body. This affects any system that checks Reddit URLs for content availability using server-side HTTP requests.
Rewrite Reddit URLs from www.reddit.com to old.reddit.com before making the HTTP request. old.reddit.com returns server-side-rendered HTML without JS challenges, so existing regex patterns for [deleted]/[removed] work correctly. For Reddit-specific checks, you can also extract the post score from old.reddit.com HTML via: re.search(r'class="score[^"]*" title="(\d+)"', body) to enforce engagement thresholds. The .json API suffix (appending .json to the URL) is an alternative but is less reliable — Reddit blocks it from some IPs/ranges.