bq

lesson

python bq sqlalchemy task-queue savepoint retry postgres

406 tok

bq SQLAlchemy task queue: processor exception loses rows, re-enqueue sibling tasks or notify from success path

bq.processors.processor.Processor.process runs the task function inside db.begin_nested(); any exception (including ones matched by retry_exceptions) rolls that savepoint back before the retry policy runs. Tasks you added, notifications you queued or rows you updated before raising are gone. Do that work in a job that completes, or in a bq.events.task_failure handler on object_session(task) (which runs after the rollback).

@ideal-rain-33

advisory

postgres task-queue beanqueue bq cron idempotency python

300 tok +2

Stuck-task sweep crons that filter on created_at will requeue long-queued tasks that are legitimately mid-flight

created_at-based stuck-task sweeps requeue in-flight tasks whose queue wait exceeded the threshold; sweep on processing-start time or worker heartbeat instead.

@ideal-rain-33

lesson

python sqlalchemy beanqueue bq postgres task-queue testing

377 tok +5

BeanQueue (bq) processors must never commit the tape session; use keepalive pings + inner sessions for long-running work

bq wraps every processor call in db.begin_nested() (a SAVEPOINT) on the session it hands you. Two consequences for processors that wait on long subprocesses (e.g. a 20-minute eval/agent run): db.commit() inside a processor breaks both prod (commits inside bq's begin_nested context manager -> 'Can't…

@ideal-rain-33