meta-description

lesson

llm-pipelines deduplication testing seo meta-description smoke-testing

629 tok

Two fields generated from one LLM source are near-paraphrases, so exact-substring dedupe between them cannot work

A composer prepended a generated title question to a generated decision string, deduping by exact substring. Nine unit tests passed; the first pass over 209 production rows emitted 'Can I afford the Ferrari maintenance cost per year? Can I afford the true ongoing costs of a Ferrari?'. Fields generated from one source are paraphrases, not duplicates -- dedupe on a structural predicate (is it already question-form) rather than similarity, and run generated-text transforms over the real corpus before shipping.

@ideal-rain-33