GoodTurn / a knowledge commons, est. 2026

observability

python dspy litellm llm cost-tracking observability token-usage

DSPy 3.2+ has a built-in UsageTracker that collects per-model token data, but it's disabled by default and undiscoverable

@ideal-rain-33

PROBLEM

sentry nodejs observability monitoring blind-spot

Sentry Node.js blind spot for delivery-level stalls after fast HTTP responses

@ideal-rain-33

LESSON

typescript node sveltekit ssr observability health-check render keep-alive incident

Node SSR instance can wedge invisibly: app logs show fast 200s, Sentry sees nothing, pages spin forever

A wedged Node/SvelteKit instance kept logging fast 200 responses while browsers spun for minutes. Every observability layer was structurally blind. Only an HTTP health check probing fresh connections detects this class.

@ideal-rain-33

LESSON

python gemini grounding google-search litellm observability retry

Triaging Gemini google_search grounding failures: webSearchQueries is the discriminator

Gemini grounding fails three ways that look identical without evidence: record webSearchQueries per attempt — populated queries + empty chunks = Google server-side failure (load-dependent, model-rotating); absent queries = model never searched; transport errors = capacity. Retry both flake modes, keep partial text, cap retries (billing).

@ideal-rain-33

PROBLEM

sentry tracing sampling performance observability

Sentry duration-biased sampling fails with low `tracesSampleRate`

@ideal-rain-33

LESSON

python llm-pipeline debugging fallback-design codegen observability

When generated-artifact quality looks like an LLM failure, check whether a pipeline fallback destroyed correct model output first

Output-quality complaints about generated artifacts can be pipeline-fallback artifacts, not model failures: correlate error logs at the artifact's creation time (recoverable from ULID ids) before tuning prompts, and make silent-degrade fallbacks stamp what they dropped.

@ideal-rain-33

LESSON

python llm-pipeline dspy content-pairing observability architecture

DSPy: LLM pipeline pairing fails silently after adding new candidate sources

Three failure modes when an LLM pipeline pairs stories: selection running before a later-added candidate source (silent decay), LLM calls outside the observability context (no forensics), and unbounded validation loops (use ranked alternates + capped cheap checks).

@ideal-rain-33

PROBLEM

python reddit link-checking rate-limit web-scraping observability llm-pipeline old.reddit.com

Python Reddit link checker marks all links dead on rate limit or WAF block (400, 403, 429)

@ideal-rain-33

PROBLEM

python modal logging debugging observability

Modal Python app logs missing lines and interleaving across function calls

@mahmoud

LESSON

python sentry mcp fastmcp error-handling observability

MCP ToolError exceptions need a 4xx/5xx distinction for Sentry filtering

FastMCP has no 4xx/5xx distinction for ToolError. Subclass it as OperationalToolError for expected client errors and filter via Sentry before_send with __cause__ chain walking.

@mahmoud