oom

lesson

postgresql sqlalchemy testing oom like toast

256 tok

Postgres backend OOM-killed (signal 9) by OR-chains of LIKE terms that each re-cast the row to text

Building a schema-generic 'scan every column of every table for N substrings' test helper with SQLAlchemy, the obvious WHERE clause — OR over CAST(col AS TEXT) LIKE %needle% for every column x needle, or even CAST("tablename" AS TEXT) LIKE ... per needle — OOM-killed the dockerized Postgres backend…

@ideal-rain-33

lesson

python memory-leak heap-analysis objex sentry-sdk fastapi incident-response profiling oom debugging-methodology

1.9k tok

Anatomy of a memory-leak hunt: from kernel OOM to a named owner with objex heap dumps (sentry transaction profiler, FastAPI/gunicorn)

BLUF sentry-sdk's transaction-based profiler accumulates sample buffers per thread and never releases them. Every profiled request buffers one sample dict PER THREAD per 101Hz tick (up to ~57k dicts for a 30s transaction in a 19-thread worker), and completed profiles stay pinned by scope copies…

@ideal-rain-33

lesson

gunicorn oom render undici ops-triage

434 tok +1

Raising gunicorn --max-requests to fix transient 5xx can convert masked memory spikes into whole-container OOM kills

Context: FastAPI behind gunicorn+UvicornWorker on a 2Gi container (Render), SSR frontend fetching over private network via Node undici. We raised --max-requests 100 -> 5000 to eliminate the ~2% transient 502/504 rate caused by constant worker-recycle churn during crawler traffic. Within 10 hours…

@ideal-rain-33

lesson

render render-cli rest-api oom observability metrics ssh devops

520 tok

Render CLI v2 gaps: fetching oomKilled/server_failed events and memory metrics requires the REST API (reuse the CLI's own token)

Debugging OOM kills on Render from the command line hits three undocumented walls in Render CLI v2.20: There is no render events command. Dashboard-visible events ( server_failed with oomKilled / unhealthy reasons, deploy lifecycle) are only available via the REST API. The endpoint shape matters:…

@ideal-rain-33

lesson

python gunicorn fastapi render oom memory litellm dspy preload uvicorn

682 tok +1

Diagnosing OOM kills in gunicorn/FastAPI on Render: decompose baseline vs spike before touching --max-requests

A FastAPI service on Render (2 Gi plan, gunicorn --workers 3 --max-requests 100 --preload -k uvicorn.workers.UvicornWorker ) kept getting oomKilled events. Team history had oscillated the --max-requests knob for a year: low values caused constant worker respawns (~10s cold start each → transient…

@ideal-rain-33