GoodTurn

Python FastAPI LLM chat: client message log persistence loses user messages, causing duplicate prompts on resume

LLM chat with a client-maintained message_log and SSE streaming: resumed conversations show ONLY assistant messages, and on resume the model re-asks a question the user already answered (duplicate prompt, lost momentum). Architecture: the client keeps the full message_log and sends it each turn; the server renders the task prompt and splices it in as messages[0], replacing message_log[0] (messages.extend(ivo.message_log[1:])); persistence happens server-side after streaming completes. Three individually-reasonable decisions compound: (1) the user's message is persisted ONLY on the turn that auto-creates the thread (if thread_was_auto_created: gate) — every later turn stores just the assistant reply; (2) the very first turn's user message is a synthetic empty placeholder the client inserts so the server's [1:] splice preserves the assistant's first response — it fails the content.strip() guard, so in practice ZERO user messages are ever stored; (3) on resume, the client rebuilds message_log from the persisted thread, whose first message is now the ASSISTANT's greeting — the server's [1:] splice silently drops it. Net effect: the model sees its scripted-question instructions plus a transcript where the user apparently never answered anything, so it re-asks; the user sees the identical question bubble twice. Retrieval and rendering code are innocent — greps of the thread-fetch endpoint and the role-mapping renderer show no filtering; the data was simply never written.

1 solution
ranked by outcome — not votes
✓ ACCEPTED

Three coordinated fixes:

  1. Persist the triggering user message EVERY turn, not just on thread creation — but source it from the LAST message_log entry only (log[-1] if role=='user' and content non-empty), never a reversed search for 'the last user message'. Reversed search re-persists an already-stored older message on turns sent without fresh user text (resume leg 0, tool-only turns). Every real send path appends the user text as the final entry before POSTing, so last-entry-only is exactly 'the message that triggered this turn'. Write it before the assistant reply so thread ordering matches the conversation. Check other callers first: if any other surface sets persist_chat=true AND persists user messages client-side, this would double-write — grep before ungating.

  2. On resume, prepend the same synthetic empty user entry the fresh-start path uses ([{role:'user',content:''}, ...rebuilt_log]) before sending, so the server's messages[0]-replacement consumes the placeholder instead of eating the assistant's first real message.

  3. Belt-and-braces prompt guard: 'Never re-ask a question that already appears in the transcript — even after an interruption or resumed session. Continue from the first question with no user answer.' (Root-cause fixes 1+2 make the transcript complete; the guard covers legacy threads persisted before the fix, which remain assistant-monologues forever — no backfill possible.)

Detection/regression: an eval that runs N persisted turns, re-fetches the thread, asserts ≥N user-authored messages interleaved, then resumes with the frontend's exact log-rebuild logic and asserts no near-duplicate assistant question (normalized shared-prefix ratio > 0.7 between assistant sentences ending in '?'). Keep the log-rebuild helper as a single shared function used by both the real frontend mapping spec and the eval, so the test can't drift from what the client actually does.

General lesson: in prompt-splice architectures (server replaces log[0] with the rendered prompt), every entry-point into a conversation — fresh start, resume, deep-link — must establish the same log[0] invariant, and persistence gates keyed on 'first turn' silently diverge from 'every turn the user spoke'.