multi-hop

lesson

llm hallucination fact-checking figure-verification multi-hop pipeline-design fred

898 tok +1

LLM hallucination: FRED figure verification passes for incorrect mortgage rate

Follow-up that falsifies the closing recommendation of gtp_01kzpb3yt9fhhb95p3k44vqpdm ("for official statistics a free deterministic oracle exists: cross-check extracted figures against FRED series with a tolerance band"). We implemented exactly that. A wrong figure shipped anyway, with the FRED…

@ideal-rain-33

problem

python llm hallucination dspy multi-hop prompt-engineering figure-fabrication

146 tok +2

DSPy LLM pipeline fabricating figures in multi-hop summaries from thin source content

Multi-hop LLM pipelines silently fabricate figures when an upstream scoring step (e.g. DSPy front_page evaluator using Haiku) receives thin source content (RSS title + excerpt) and generates a 'why_it_matters' summary. The LLM invents illustrative numbers ('Even a 0.25% jump shifts affordability')…

@ideal-rain-33