When an LLM-powered news digest pipeline (e.g. daily financial vibes from grounded search) leads with the same story across multiple editions because the underlying data point dominates the news cycle for days, the fix belongs in the writer stage, not the search/ranking stage.
Wrong approach: Modify the grounded search prompt to deprioritize stories it has seen before. This corrupts the ranking signal — the story IS genuinely the biggest news; suppressing it means the search returns less accurate results.
Right approach: Query prior editions' lead headlines, pass them as recurring_macro_leads context to the writer, and instruct the writer to:
Implementation pattern:
# 1. Query helper: deduplicated lead headlines from recent editions
def _recent_leads(db_session, edition_date, n_days=2) -> list[str]:
# Filter by origin, date range (>= cutoff, < current), order by date DESC, slot DESC, rank ASC
# Take first (lowest rank) per (date, slot) pair, deduplicate headlines
...
# 2. Thread through to the writer's context JSON
developing_story = json.dumps({
'recent_daily': recent_daily,
'recurring_macro_leads': recent_leads or [],
})
# 3. Writer prompt instruction (DSPy Signature docstring):
# 'If developing_story lists recurring_macro_leads and the biggest macro story
# matches one of them, LEAD with the SECOND macro story as fresh news and
# weave the recurring story in as backdrop context.'Key design decisions:
recent_leads list is recorded in the debug manifest for traceabilityrecent_leads: list[str] | None = None) ensures backward compatibility