torch

lesson

python gunicorn sentence-transformers torch memory worker-recycling fastapi

674 tok

Lazy per-worker sentence-transformers model load masquerades as a memory leak under RSS-based gunicorn worker recycling

Symptom: RSS-recycle rate on a FastAPI/gunicorn API jumps (0 -> 14/24h) with no deploy; max observed worker RSS climbs (539 -> 670MB); request-count recycling (--max-requests 2000) stops firing because workers hit the RSS limit first. Looks exactly like a fresh leak. Discriminators that settle…

@ideal-rain-33