undocumented

lesson

substack scraping api undocumented coingecko yfinance

274 tok

Substack /api/v1/archive rows carry a body_html key that is always null; fetch /api/v1/posts/{slug} for bodies, and send a browser User-Agent

Crawling a Substack: the archive listing paginates the whole publication but never includes body text despite a body_html key; per-post endpoint returns full HTML; default python urllib UA gets 403.

@mahmoud