Swapping a grounded-search provider silently breaks downstream attribution extraction that depended on the old provider's response shape

Context

An LLM content pipeline did grounded web search in two phases: Phase A calls a search-capable model and captures its output; Phase B is a structured extractor that reads Phase A's text and emits {headline, why_it_matters, source_name} per story. source_name is rendered as the public citation label: [CNBC](https://...).

The provider for Phase A was swapped (Gemini grounding -> Anthropic web_search passthrough) in the same release as an unrelated figure-integrity hardening. All tests passed. The very first production edition shipped [Grounded news summary (August 11, 2026)](https://themortgagereports.com/...) to readers as a source attribution.

Root cause

The two providers return structurally different things and the extractor's prompt silently depended on the difference:

• Gemini grounding returns per-article chunks carrying titles and outlet identity. Phase B could read an outlet name off the material.
• Anthropic web_search returns one chatty narrative that opens "I'll search for today's major economic and financial news..." followed by prose sections. There is no per-story outlet name anywhere in it.

Asked for "the primary outlet or releasing agency" with no outlet in its input, the extractor did the most reasonable available thing: it named the document it was reading. 0 of 3 stories produced a usable outlet label. The one correct label in the output came from a different, non-grounded code path.

Second-order damage (the part that cost the most)

The downstream writer had a style rule requiring a named-outlet citation. Faced with two items labelled Grounded news summary (...), it did not fail loudly. It:

1. Dropped both items as leads - including the search's own #1-ranked story, which never reached readers at all.
2. Spliced a figure it wanted from a placeholder-labelled item onto a different item's real outlet citation, publishing [CNBC](...) reported ... 30-year fixed mortgages sit at 6.763% where the CNBC page contains no such figure.

So one degraded field in an intermediate extraction became a factual misattribution on a public page, laundered through an LLM that was trying to satisfy its style constraints.

Lessons

1. A provider swap is a schema change even when the interface is str -> str. Prompts encode assumptions about the shape and content of upstream text, not just its type. Diff a real sample of both providers' raw output before cutting over, not just the parsed result.
2. Validate extracted labels against a predicate, not just non-emptiness. source_name was non-empty, plausible, and grammatical. Cheap guard: reject labels containing summary/briefing/search/grounded, or a bare date; fall back to deriving the publisher from the resolved URL's domain.
3. Downstream LLMs launder upstream degradation into confident falsehoods. A rule-following writer given an unusable citation will not error - it will substitute a usable-looking one. Any place a model chooses which of several provided sources to cite needs a post-generation check that each figure appears in the item whose link is actually cited.
4. Verification gates validate the pairings the pipeline built, not the ones the writer published. Every integrity field read green (figure_verified: true, factcheck verdict ok, per-chunk citation_verified) because they all ran before the writer. Put at least one assertion after the last generative step.
5. Watch for the gate that checks an unshipped value. A FRED cross-check validated the figure in the story headline while the prose shipped a different figure from a regenerated summary field. Green verdict, zero reader coverage. Fact-check the string that ships.

Detection recipe

# 1. non-entity attribution labels
labels = re.findall(r'\[([^\]]+)\]\((https?://[^)]+)\)', published_text)
bad = [n for n, _ in labels if re.search(r'(?i)summary|briefing|grounded|search|^\W*\d{4}', n)]

# 2. cross-item figure splice: every figure in a cited sentence must live in
#    the source item whose link that sentence cites
for name, url in labels:
    item = next(i for i in macro_items if i['link'] == url)
    for fig in re.findall(r'\d[\d,.]*%?', sentence_citing(url)):
        assert fig in item['text'], f'{fig} spliced onto {name}'