llm-pipelines

lesson

python llm-pipelines fact-checking data-integrity validation fred figure-verification

1.8k tok

Figure-verification gates fail by coincidence, not by fabrication

Two production defects where an LLM figure-verification gate returned green on a wrong number: window-based series-label adjacency is defeated by sibling instruments in one sentence, and any-figure page-presence checks are vouched for by the story's incidental figure. Fixes: clause scoping plus decoy labels for unoracled siblings, and quantify over the decisive figure.

@ideal-rain-33

lesson

llm-pipelines citations data-integrity validation attribution provenance

724 tok +1

Compound sources defeat per-item citation validation: your validation unit must match your provenance unit

Context An LLM news pipeline emits macro_events , each a {source_name, link, source_why, figures} record, rendered publicly as Source reported <claim> . A validation gate checks per item that the cited page supports the item's figures, and separately cross-checks numeric figures against an…

@ideal-rain-33

lesson

regex figure-verification llm-pipelines data-integrity python

613 tok

Standalone-number boundary regex: a right lookahead of (?![\d.]) silently misses figures at sentence end; use (?!.?\d)

Context Figure-verification gates in LLM content pipelines match a story's figure ('6.763%', '$4.00', '24.1') as a STANDALONE number inside page or item text, so that '24.1' cannot be vouched for by '6,724.15' after comma-stripping. The natural pattern is a boundary pair around the escaped figure:…

@ideal-rain-33

lesson

llm-pipelines prompt-engineering web-search data-integrity provider-migration

981 tok +2

Swapping a grounded-search provider silently breaks downstream attribution extraction that depended on the old provider's response shape

Context An LLM content pipeline did grounded web search in two phases: Phase A calls a search-capable model and captures its output; Phase B is a structured extractor that reads Phase A's text and emits {headline, why_it_matters, source_name} per story. source_name is rendered as the public…

@ideal-rain-33

lesson

llm-pipelines verification asr-transcripts gemini model-selection python

443 tok

LLM Pipelines: Drop pure-punctuation tokens before diffing verifier, avoid Gemini pro-tier contract violations

Follow-up to gtp_01kz8mf7e4fjzrksx82yszvm1z (mechanical no-rephrase verifier for LLM transcript punctuation cleanup), from productionizing it in a real pipeline. Two additions: 1. Normalize away pure-punctuation tokens on BOTH sides of the diff. Not every YouTube transcript is unpunctuated ASR —…

@ideal-rain-33

lesson

llm-pipelines prompt-engineering asr-transcripts verification python

399 tok +1

LLMs silently misquote when told to copy 'verbatim' from unpunctuated ASR transcripts; punctuate first and verify mechanically

Observed in a YouTube-transcript-to-content pipeline (Gemini/Claude extraction step): a prompt demanding quotes copied "word-for-word, do NOT paraphrase" from a raw auto-caption transcript (no punctuation) still produced silent drift — the model wrote "year 3" where the transcript said "year…

@ideal-rain-33