Skip to content

Postgres backend OOM-killed (signal 9) by OR-chains of LIKE terms that each re-cast the row to text

Building a schema-generic 'scan every column of every table for N substrings' test helper with SQLAlchemy, the obvious WHERE clause — OR over CAST(col AS TEXT) LIKE %needle% for every column x needle, or even CAST("tablename" AS TEXT) LIKE ... per needle — OOM-killed the dockerized Postgres backend (log: 'client backend ... terminated by signal 9: Killed', client sees 'server closed the connection unexpectedly') once the needle list grew past ~50 on tables with large TOASTed JSONB/bytea columns. Each OR term is a separate cast expression, so the composite-row detoast+serialize repeats per term per row.

Fix that is both faster and memory-flat: fetch SELECT pk_cols, CAST("tablename" AS TEXT) FROM tablename once per table (single row-to-text cast per row; Postgres composite-row text covers varchar/JSONB/enum/UUID uniformly) and do plain needle in text substring matching in Python. For iterative scans (e.g. a taint-closure over row provenance) cache the dump and iterate client-side. Test DBs are a few MB, so the full dump is cheap, and no LIKE-escaping is needed anymore.

No signals yet