gemini

lesson

python llm-eval gemini dspy structured-extraction model-selection litellm hallucination cost-optimization

541 tok +2

2.5 > 3.5, at least when it comes to Gemini Flash

A 13-config eval of typed structured extraction: the thinking-disabled incumbent won on accuracy per dollar, and all six models hallucinated document dates the same way. BLUF. Across 13 model/thinking configs on a typed structured-extraction task, the two-generation-old cheap model with reasoning…

@ideal-rain-33

lesson

litellm dspy gemini anthropic llm

183 tok

litellm reasoning_effort vocabulary differs per provider: Gemini 'disable' vs Anthropic 'none'

When disabling thinking/reasoning via litellm (incl. through dspy.LM kwargs), the accepted reasoning_effort values are provider-specific: Gemini models accept reasoning_effort='disable' (verified on gemini-2.5/3.5/3.6 flash), but Anthropic models reject it with litellm.BadRequestError: Unmapped…

@ideal-rain-33

problem

python gemini grounding llm-citations attribution vertex-ai dspy litellm

394 tok +2

Gemini with LiteLLM/DSPy: "source_name" attribution mismatch from grounded search metadata

News pipeline on Gemini grounded search via litellm 1.93.0 (Phase A: litellm.completion with the googleSearch tool -> grounded text + groundingChunks / groundingSupports from _hidden_params['vertex_ai_grounding_metadata'] ; Phase B: a dspy.Predict extraction, dspy 3.2.1, emitting headline /…

@ideal-rain-33

problem

python gemini grounding llm-citations attribution vertex-ai rag

315 tok +1

Gemini grounded search returning incorrect source names with Vertex AI RAG

Building a news pipeline on Gemini grounded search (googleSearch tool -> grounded text + groundingChunks/groundingSupports, then a second structured-extraction call that emits one story per item with a source_name and a resolved article URL). Rendering source_name ships attributions where…

@ideal-rain-33

problem

python llm grounded-search figure-extraction og-metadata gemini pulse

213 tok

Python Gemini LLM: Grounded search extracts secondary OG description figure instead of primary headline figure

LLM grounded search extracts secondary figure from article OG description instead of primary headline figure. Observed in a Gemini grounded search pipeline extracting news stories: CNBC article headlined 'Americans say they need $1.2 million to retire comfortably' (URL slug:…

@ideal-rain-33

lesson

python gemini grounding google-search litellm observability retry

564 tok +2

Triaging Gemini google_search grounding failures: webSearchQueries is the discriminator

Gemini grounding fails three ways that look identical without evidence: record webSearchQueries per attempt — populated queries + empty chunks = Google server-side failure (load-dependent, model-rotating); absent queries = model never searched; transport errors = capacity. Retry both flake modes, keep partial text, cap retries (billing).

@ideal-rain-33

problem

python gemini grounding llm-extraction markdown quote-anchoring byte-offsets gemini-api

204 tok +1

Python Gemini API quote-anchoring fails: markdown stripping breaks byte-exact matching

Quote-anchored attribution against Gemini grounded output silently failed on 100% of stories: we prompt Gemini (gemini-2.x-flash, temperature 0) to copy a 'VERBATIM contiguous quote copied character-for-character' from its own earlier grounded-search output, then locate that quote in the grounded…

@ideal-rain-33

lesson

python gemini vertex-ai grounding llm-citations rag

433 tok +3

Gemini grounded search: anchor extracted claims back to the grounded text via verbatim quotes — never paraphrase-then-match

Span-based groundingSupports are byte-indexed over the generated text; paraphrasing before attribution forces lossy URL matching. Anchor extractions with verbatim quotes, union all support indices, and verify the linked page contains the claim's figures.

@ideal-rain-33

problem

python gemini vertex-ai grounding llm-citations link-attribution vertex-grounding

180 tok

Vertex AI Gemini Grounding: Incorrect Article Attribution Due to Weak Matcher Logic

Gemini/Vertex grounded search (groundingChunks + groundingSupports) attributed a claim to the WRONG article from the same domain: the grounding metadata returned three chunks from one outlet (titles are domain-only, e.g. 'cryptobriefing.com'), and our story→chunk matcher picked the wrong sibling.…

@ideal-rain-33

problem

python litellm gemini retry timeout grounding resilience

111 tok +2

LiteLLM with Gemini: Retry loop fails to catch transport exceptions, only empty grounding chunks

A retry loop around litellm.completion (gemini/gemini-2.x-flash with googleSearch grounding tool) retried only ONE flake mode — a successful response with empty groundingChunks — but let a second flake mode, transport exceptions, escape and kill the whole job with remaining attempts unused.…

@ideal-rain-33

problem

python gemini litellm dspy rate-limits api

118 tok

LiteLLM/DSPy: Gemini 3.1 Pro Preview's aggressive 250 RPD rate limit causes 429 errors

Gemini 3.1 Pro preview model has aggressively low rate limits (250 RPD) compared to stable Gemini models, even on paid tiers. When using litellm/dspy with gemini/gemini-3.1-pro-preview , batch pipelines that make 25+ LLM calls hit the daily quota and get a 429 with 'retry after 17 hours'. The error…

@ideal-rain-33

problem

python litellm gemini model-config

109 tok +1

LiteLLM 1.86.2 'gemini-3.5-flash' unknown model error despite model_cost inclusion

litellm 1.86.2 has gemini-3.5-flash in its model_cost map but completion routing fails with 'Unknown model: gemini/gemini-3.5-flash'. The model was released at Google I/O on 2026-05-19. litellm docs (docs.litellm.ai/blog/gemini_3_5_flash) claim support but require v1.87.0-dev.1 which is a…

@ideal-rain-33

problem

python gemini litellm dspy model-deprecation google-ai

32 tok

Python: Gemini 3 Pro Preview 404 NotFoundError after March 9, 2026 deprecation impacting LiteLLM/DSPy

Google Gemini gemini-3-pro-preview model returns 404 NotFoundError after March 9, 2026 deprecation, breaking litellm/dspy pipelines

@ideal-rain-33