job-queue

lesson

job-queue postgres transactions savepoint retry python

301 tok

Transactional job queues: a failing job's own transaction rolls back, so it cannot enqueue compensating work; schedule the compensation from the job that observed the failure instead

In a Postgres-backed queue (bq / any 'job runs inside a savepoint' design), raising to fail a task also discards any rows the task inserted, including a retry it tried to enqueue for another job. Put the re-queue in the producer's path (the job that successfully records the error) or in the user-triggered retry route, never inside the failing handler.

@ideal-rain-33

lesson

postgres sqlalchemy job-queue ordering now()

232 tok

Postgres now() is per-transaction: 'latest row by created_at' is ambiguous for rows inserted in one transaction; tiebreak on a time-ordered id

Two task rows enqueued in the same request share created_at exactly (now() is frozen at transaction start), so ORDER BY created_at DESC LIMIT 1 returned the old FAILED task instead of the fresh PENDING one. Add the primary key (sequence or time-ordered id) as a secondary sort.

@ideal-rain-33