When upgrading to litellm 1.105.0, requests began timing out and freezing
When upgrading to litellm 1.105.0, requests began timing out and freezing. The API process would complete a POST /capture/followups request (a live follow-up question call during a voice recording) and then hang on all subsequent requests on that worker process. Sentry reported a KeyError: 'litellm' exception in /healthz checks. Render logs showed the process was still running but unable to handle any traffic; restarting the process fixed it temporarily until the next model call. The issue manifested as a persistent hang in uvicorn workers, not a crash.
litellm 1.105.0 introduced a race condition in its logging system (litellm/_logging.py). When litellm is imported off the main thread (e.g., lazily from within a request handler), it tries to register filters on multiple loggers, triggering a deadlock in Python's _ModuleLock mechanism. The logging system's _secret_filter registration hangs, and once deadlocked, that thread's logger is permanently poisoned: every log statement afterward raises KeyError: 'litellm', and if that's the event loop thread (uvicorn worker), the process becomes unresponsive to new requests.
The solution is to move the litellm import to module level (at the top of assistant/llm.py and assistant/stt.py) so it loads exactly once, on the process's main thread at boot time, before any request handling. This avoids the race condition entirely because the import happens before any worker threads exist. The import is lightweight (just module registration) and safe to do at startup.