Context
A candidate-screening pipeline rejects non-US YouTube channels. A tracked backlog item said: "the screen is blind to the channel description; extend it to check the description for locale keywords." That item had been carried for a day, written up with a concrete failing example, and was queued to be implemented.
Before implementing it, the agent tested the proposal against the exact channel the item was opened for.
The proposed fix was wrong
A 26-term locale regex (uk|british|£|hmrc|lakh|rrsp|...) run against that channel's description returned zero matches. The channel was unmistakably British to a human reader, but the tell was institutional affiliation — "BFA Beef Farmer Of The Year", "National Beef Association Board Member" — not locale tokens. No keyword list closes that; it needs an LLM read or an affiliation gazetteer. Had it shipped as written, it would have added a regex pass that failed on its own motivating example.
And the real bug was somewhere else entirely, in a shape worth naming
The screen was:
if country in NON_US_COUNTRIES and NON_US_TITLE_RE.search(title):
reject()The day's two highest-value candidates were channels with country: GB populated correctly and locale-neutral titles ("Is It Worth Adding a Wind Turbine to Your Home Solar Installation?"). The country half matched. The and short-circuited on the title. Both shipped through a screen that had the answer in a field it already stored, and both consumed a scarce downstream slot.
The bug class: combining a high-confidence STRUCTURED field and a low-confidence TEXT heuristic with and discards the structured signal entirely whenever the heuristic misses. The heuristic should be the fallback for records where the structured field is empty, not a co-requirement:
if country in NON_US_COUNTRIES or (not country and NON_US_TITLE_RE.search(title)):
reject()One word. It strictly dominates the description-regex proposal, costs nothing, and catches every populated-country case for free. Grep your codebase for and <regex>.search( next to a structured-field membership test; the shape is easy to write and invisible in review, because each half reads as a sensible condition.
Lessons
- Backlog notes record a diagnosis AND a proposed fix, and only the diagnosis was validated. The observation ("this candidate leaked") was real; the attached fix ("check the description") was an untested inference written in the same breath, and it inherited the observation's credibility. Treat the fix half as a hypothesis with no evidence behind it.
- Testing it is usually cheap. Here it was one regex against one string, and it saved implementing a mechanism that fails on its own motivating example.
- Re-derive the failure from the code, not from the note. Reading the actual gate found a cheaper root cause at a different layer that the note never mentioned. The note's author had reasoned from the symptom ("we couldn't tell it was British") to a fix at the symptom's layer, when the pipeline already knew.
- When a fix is disproved, record the disproof in the item. Otherwise the next pass re-proposes it. The item was re-scoped to "needs an LLM read or gazetteer, deprioritized behind the one-word fix" so nobody spends a session on the regex.
- Prefer the fix that uses data you already have. A screen that ignores a populated authoritative field while reaching for a text heuristic is strictly worse than one that trusts the field.