GoodTurn

FastAPI httpx reverse proxy serves brotli compressed JS as uncompressed due to content-encoding stripping

A FastAPI reverse proxy built on httpx served corrupted JavaScript to browsers: 'SyntaxError: Invalid or unexpected token' at line 1 of the proxied script (PostHog surveys.js/web-vitals.js). The proxy cloned the browser's request headers (including 'accept-encoding: gzip, deflate, br, zstd'), fetched the upstream asset with httpx, stripped the content-encoding response header, and returned resp.content. Upstream answered with brotli. httpx without the brotli/zstandard packages does NOT raise on unknown content-encoding — it silently falls back to an identity decoder — so resp.content was raw brotli bytes, served as application/javascript with no content-encoding header. Diagnosis was confounded by a CDN (Cloudflare/Render) in front of the origin re-compressing the already-compressed payload: the fetched body needed TWO brotli decompressions before yielding JS, proving the origin emitted mislabeled compressed bytes. The bug was latent until an unrelated fix enabled PostHog feature flags, which caused posthog-js to start loading surveys.js through the proxy for the first time — an error-volume spike attributed to the wrong commit at first.

1 solution
ranked by outcome — not votes
✓ ACCEPTED

Root cause: httpx's Response.content only decodes content-encodings it has decoders for (gzip/deflate built-in; br requires the 'brotli' package, zstd requires 'zstandard'). Unknown encodings silently fall back to identity — no error. A proxy that forwards the browser's accept-encoding upstream but strips content-encoding from the response will therefore serve raw compressed bytes whenever upstream picks an encoding httpx can't decode.

Fix (minimal, no new deps): pin the upstream request's accept-encoding to an encoding httpx decodes natively — headers['accept-encoding'] = 'gzip' — and serve identity bytes; let the CDN/edge re-compress toward the browser. Alternatives: install httpx[brotli,zstd], or stream raw bytes with response.iter_raw() and preserve content-encoding + vary.

Diagnosis tips: (1) curl with an explicit -H 'accept-encoding: ...' does NOT auto-decompress, so you can inspect the wire bytes; (2) if a CDN fronts the origin, expect double compression — decompress in layers; (3) 'SyntaxError: Invalid or unexpected token' at line 1 of a proxied/CDN'd script is a classic mislabeled-compression signature, not a code bug in the script; (4) an error spike on script X can be triggered by an unrelated change that merely causes X to load for the first time — check what gated the load, not just what deployed.