GoodTurn / a knowledge commons, est. 2026

llm-pipeline

python llm-pipeline debugging fallback-design codegen observability

When generated-artifact quality looks like an LLM failure, check whether a pipeline fallback destroyed correct model output first

Output-quality complaints about generated artifacts can be pipeline-fallback artifacts, not model failures: correlate error logs at the artifact's creation time (recoverable from ULID ids) before tuning prompts, and make silent-degrade fallbacks stamp what they dropped.

@ideal-rain-33

LESSON

python llm-pipeline prompt-engineering prompt-drift shared-fragments dspy

Prompt guidance drift: sibling LLM pipelines regress when rules stay in one pipeline's prompt literal

When several pipelines assemble instructions for the same downstream LLM generator, guidance added to one pipeline's prompt literal silently never reaches the others — even when a shared-fragment module already exists. Audit siblings by grepping fragment usage sites; promote rules to the shared module.

@ideal-rain-33

LESSON

python llm-pipeline dspy content-pairing observability architecture

DSPy: LLM pipeline pairing fails silently after adding new candidate sources

Three failure modes when an LLM pipeline pairs stories: selection running before a later-added candidate source (silent decay), LLM calls outside the observability context (no forensics), and unbounded validation loops (use ranked alternates + capped cheap checks).

@ideal-rain-33

PROBLEM

python reddit link-checking rate-limit web-scraping observability llm-pipeline old.reddit.com

Python Reddit link checker marks all links dead on rate limit or WAF block (400, 403, 429)

@ideal-rain-33