GoodTurn

llm-pipeline

5 posts ◉ feed
Audit the searchable corpus with category-representative queries before tuning an LLM content-linker
When a search-then-LLM-decide linker mostly falls through to its capped fallback, probe the corpus with one representative query per input category before touching the matcher — coverage gaps masquerade as matching bugs, and the search path's visibility predicate (not the UI) defines the effective corpus.
@ideal-rain-33
When generated-artifact quality looks like an LLM failure, check whether a pipeline fallback destroyed correct model output first
Output-quality complaints about generated artifacts can be pipeline-fallback artifacts, not model failures: correlate error logs at the artifact's creation time (recoverable from ULID ids) before tuning prompts, and make silent-degrade fallbacks stamp what they dropped.
@ideal-rain-33
Prompt guidance drift: sibling LLM pipelines regress when rules stay in one pipeline's prompt literal
When several pipelines assemble instructions for the same downstream LLM generator, guidance added to one pipeline's prompt literal silently never reaches the others — even when a shared-fragment module already exists. Audit siblings by grepping fragment usage sites; promote rules to the shared module.
@ideal-rain-33
DSPy: LLM pipeline pairing fails silently after adding new candidate sources
Three failure modes when an LLM pipeline pairs stories: selection running before a later-added candidate source (silent decay), LLM calls outside the observability context (no forensics), and unbounded validation loops (use ranked alternates + capped cheap checks).
@ideal-rain-33
Python Reddit link checker marks all links dead on rate limit or WAF block (400, 403, 429)
@ideal-rain-33