A review/generation feature cached its LLM output on sha256(canonical_input_json): same view content, reuse the stored review. Correct for cost, wrong as a product contract. The hash covers only the input; the producer also changes — prompt edits, model upgrades, a run where the model just misbehaved. Once content is hashed, a user-facing 'regenerate' button becomes a permanent no-op: the only escape is mutating the input, which users do not want to do just to re-roll an answer. Ours shipped a button whose sole possible response for months was 'already reviewed this exact version'.
Fix shape that keeps the cost win:
- Keep the content hash as the cache key, but make the reuse decision time-aware: cache expiry belongs to the user-initiated path only (
ensure_review(..., max_cache_age=timedelta(hours=24))); the automatic triggers (on publish, on new revision) keep a never-expiring cache so no background path pays twice. - Give privileged callers (staff/ops) an unconditional bypass — they are the ones diagnosing a bad generation, and a 24h wait makes the diagnosis loop useless.
- A cache-hit response must be actionable, not just a status enum. Return the retry timestamp alongside
cached(last-generation time + window), so the UI can say 'change the input and publish a new version, or ask again in 8 hours' instead of a dead-end 'already done'. A bare{status: 'cached'}forces the frontend to either lie about timing or hardcode the window — and a hardcoded frontend copy of a backend policy constant drifts the first time you tune it.
General rule: when the cache key covers the input but not the generator, the cache must expire on time or on a generator version, and the API must expose which. Anywhere a human can press 'try again', 'nothing happened' is a bug even when it is technically a hit.