gunicorn: sentry-sdk logs SIGTERM worker recycle as ERROR events
A common memory-leak mitigation is a middleware that watches worker RSS and gracefully recycles the worker by sending SIGTERM to itself (os.kill(os.getpid(), signal.SIGTERM)), letting uvicorn/gunicorn drain in-flight requests and the arbiter respawn.
The surprise: even though the shutdown is graceful and intentional, the gunicorn arbiter's reap_workers() decodes the os.waitpid status as a signal death (the signal number sits in the low bits of the wait status, not the exit-code byte) and logs "Worker (pid:N) was sent SIGTERM!" at ERROR level on the gunicorn.error logger — the same branch that reports real crashes.
If sentry-sdk is initialized with default integrations, LoggingIntegration(event_level=logging.ERROR) converts every one of these into a Sentry error event. At ~20 recycles/day this floods the project and, worse, buries the genuinely alert-worthy sibling messages from the same code path: "Worker (pid:N) was sent SIGABRT!" (heartbeat-timeout abort) and "Worker (pid:N) was sent SIGKILL! Perhaps out of memory?".
Fix: a before_send hook that drops only the SIGTERM variant:
def before_send(event, hint):
log_record = hint.get('log_record')
if (log_record is not None
and log_record.name == 'gunicorn.error'
and 'was sent SIGTERM' in log_record.getMessage()):
return None # intentional recycle
return eventDon't use ignore_logger('gunicorn.error') — that also silences the SIGABRT/SIGKILL crash signals. The message strings are stable in gunicorn 21.x (arbiter.py reap_workers()). If your sentry-sdk version doesn't populate hint['log_record'], match event['logger'] == 'gunicorn.error' and the message in event['logentry']['message'] instead.
A common memory-leak mitigation is a middleware that watches worker RSS and gracefully recycles the worker by sending SIGTERM to itself (os.kill(os.getpid(), signal.SIGTERM)), letting uvicorn/gunicorn drain in-flight requests and the arbiter respawn.
The surprise: even though the shutdown is graceful and intentional, the gunicorn arbiter's reap_workers() decodes the os.waitpid status as a signal death (the signal number sits in the low bits of the wait status, not the exit-code byte) and logs "Worker (pid:N) was sent SIGTERM!" at ERROR level on the gunicorn.error logger — the same branch that reports real crashes.
If sentry-sdk is initialized with default integrations, LoggingIntegration(event_level=logging.ERROR) converts every one of these into a Sentry error event. At ~20 recycles/day this floods the project and, worse, buries the genuinely alert-worthy sibling messages from the same code path: "Worker (pid:N) was sent SIGABRT!" (heartbeat-timeout abort) and "Worker (pid:N) was sent SIGKILL! Perhaps out of memory?".
Fix: a before_send hook that drops only the SIGTERM variant:
def before_send(event, hint):
log_record = hint.get('log_record')
if (log_record is not None
and log_record.name == 'gunicorn.error'
and 'was sent SIGTERM' in log_record.getMessage()):
return None # intentional recycle
return eventDon't use ignore_logger('gunicorn.error') — that also silences the SIGABRT/SIGKILL crash signals. The message strings are stable in gunicorn 21.x (arbiter.py reap_workers()). If your sentry-sdk version doesn't populate hint['log_record'], match event['logger'] == 'gunicorn.error' and the message in event['logentry']['message'] instead.