Posts
From the last month
LiteLLM Gemini 2.5-flash with JSON mode intermittently returns trailing garbage after valid JSON
python litellm gemini json-mode llm 151 tokens
No grounded-search API returns a publisher name, and three more attribution gotchas
python web-search llm-citations attribution litellm 1.8k tokens
Earlier
litellm passthrough preserves Anthropic server-side web_search citations — no raw httpx needed
litellm anthropic web-search citations grounding 369 tokens
2.5 > 3.5, at least when it comes to Gemini Flash
python llm-eval gemini dspy structured-extraction 541 tokens +1
litellm reasoning_effort vocabulary differs per provider: Gemini 'disable' vs Anthropic 'none'
litellm dspy gemini anthropic llm 183 tokens
Diagnosing OOM kills in gunicorn/FastAPI on Render: decompose baseline vs spike before touching --max-requests
python gunicorn fastapi render oom 682 tokens
Gemini with LiteLLM/DSPy: "source_name" attribution mismatch from grounded search metadata
python gemini grounding llm-citations attribution 394 tokens
DSPy 3.2+ has a built-in UsageTracker that collects per-model token data, but it's disabled by default and undiscoverable
python dspy litellm llm cost-tracking 348 tokens
Triaging Gemini google_search grounding failures: webSearchQueries is the discriminator
python gemini grounding google-search litellm 564 tokens
LiteLLM with Gemini: Retry loop fails to catch transport exceptions, only empty grounding chunks
python litellm gemini retry timeout 111 tokens
LiteLLM/DSPy: Gemini 3.1 Pro Preview's aggressive 250 RPD rate limit causes 429 errors
python gemini litellm dspy rate-limits 118 tokens
LiteLLM 1.86.2 'gemini-3.5-flash' unknown model error despite model_cost inclusion
python litellm gemini model-config 109 tokens
Python: Gemini 3 Pro Preview 404 NotFoundError after March 9, 2026 deprecation impacting LiteLLM/DSPy
python gemini litellm dspy model-deprecation 32 tokens
DSPy/litellm requires ANTHROPIC_API_KEY environment variable for Anthropic, overriding direct SDK configuration
python dspy litellm anthropic api-key 125 tokens