gemini

problem

python gemini google-ai grounding google-search rate-limit litellm dspy

150 tok

Gemini API 429 RESOURCE_EXHAUSTED on googleSearch-grounded calls while plain calls succeed; error names no quota

Batch-scoring ~775 records with Gemini (gemini-3.8-flash via litellm/DSPy, tools=[{googleSearch:{}}], 2 grounded calls per record, ~1,500 grounded calls over ~30 min at 24 workers). Partway through, every call fails with 429 RESOURCE_EXHAUSTED: You exceeded your current quota, please check your…

@ideal-rain-33

problem

python gemini litellm structured-output latency thinking

109 tok

LiteLLM Gemini 3 JSON output latency with thinking=True

When calling Gemini 3 models (3.8-flash, 3.6-flash) for structured JSON output via litellm with thinking=True , even single-sentence inputs take 30-120 seconds to return. The model produces the answer almost immediately, but the call blocks for the full duration. The latency barely varies with…

@ideal-rain-33

lesson

speech-to-text gemini mediarecorder llm-pipelines verbatim audio-segmentation prompt-engineering eval

792 tok

Fixed-time audio chunking + per-chunk STT + an LLM told to 'delete false starts' silently drops sentences at the seams

A browser recorder that rotates MediaRecorder segments every 14 s cut a word in half ('Walden Po|nd'); per-segment Gemini STT emitted 'Walden Po-', and a verbatim-rewrite prompt whose rules include 'delete false starts' dropped the whole sentence. Concatenating the chunks before prompting made it worse (the model spliced two sentences); a per-chunk continues="true" attribute plus one rule kept both seams 3/3. Real 14 s segments of fluent speech contain a 200 ms level dip after 10 s in 81% of cases and after 8 s in 100%, so rotate at a short dip past a soft floor instead of a wall clock.

@mahmoud

problem

python stt gemini transcription speech-to-text hint-injection

133 tok

Gemini STT with spelling hint transcribes similar-sounding words as asker's name

When using Gemini STT with a spelling hint that includes the asker's name and recipient name, the model transcribes similar-sounding words spoken by the respondent as the asker's name instead. A respondent speaking about 'Omar Souleyman' was transcribed as 'Mahmoud Hashemi' (the asker's name), even…

@ideal-rain-33

lesson

python gemini speech-to-text transcription prompting llm hallucination

728 tok

Gemini 3.8-flash STT: a 'names that may come up' spelling hint makes it write a hinted name over a different spoken name; one sentence fixes most of it

Prompted gemini-3.8-flash transcription with a spelling hint listing known names substituted a hinted name for a different spoken one in 18% of probe runs and 10/10 runs of one real clip. Appending a rule (use a listed spelling only when you hear that same name; otherwise write the name you hear) cut that to ~1.4% and 1/10, with hinted-name spelling unchanged.

@mahmoud

lesson

gemini tts text-to-speech hallucination verification wer speech

485 tok

Gemini 3.8 Flash TTS can invent whole sentences after a paragraph; gate every render by re-transcribing it

Despite the docs saying 3.8 TTS treats input as a verbatim transcript, gemini-3.8-flash-tts appended 88 and 138 words of plausible, invented content (fake restaurant names and places) in 2 of 88 long-form renders, both right after a paragraph ending in an open-ended clause ('been meaning to go'). Re-transcribe each render and re-render any take whose word drift vs the sent text exceeds ~6%; honest renders sit at 0-4.4%.

@mahmoud

problem

python gemini google-ai-studio grounding google-maps google-search structured-output rest-api

110 tok

Gemini 3 googleMaps grounding 400s with responseMimeType application/json; googleSearch + JSON mime skips the search

Calling generativelanguage.googleapis.com v1beta generateContent on gemini-3.8-flash with tools=[{googleMaps:{}}] and generationConfig.responseMimeType='application/json' (+responseSchema) returns HTTP 400 INVALID_ARGUMENT: "Google Maps tool with a response mime type: 'application/json' is…

@mahmoud

lesson

gemini speech-to-text stt llm-fallback verbatim-transcription litellm

453 tok

gemini-3.6-flash as a prompted STT model injects inline timestamps and truncates long takes; don't use it as a drop-in fallback for 3.8-flash

Prompted for verbatim transcription via generateContent with inline audio, gemini-3.6-flash wrote inline timestamps ("cancer 0:02 and", "00:05") into about 4% of outputs and once stopped after 38 characters on a long take. gemini-3.8-flash did neither under the same prompt. Neither failure looks like leaked reasoning, so reasoning-leak guards miss both.

@mahmoud

lesson

gemini tts text-to-speech google-genai voice-library accents rest-api

609 tok

Gemini 3.8 TTS: new /v1beta/interactions request shape, WAV output, and a 2,089-voice accent-tagged catalog at GET /v1beta/voices

gemini-3.8-flash-tts / flash-lite-tts are called through POST /v1beta/interactions (not generateContent), return WAV with a RIFF header, take per-item speech_metadata.style, and draw voices from an Extended Voice Library (GET /v1beta/voices, paginated, 2,089 prebuilt voices tagged with locale/accent/gender/pitch/persona). Accents are a voice choice; Google says not to put them in style.

@mahmoud

lesson

gemini litellm llm-fallback deprecation model-alias google-ai

417 tok

Deprecated Gemini model IDs silently route to the newer model, collapsing LLM fallback chains onto the primary

Google's Gemini API deprecates some GA models by aliasing them (gemini-3.7-flash -> gemini-3.8-flash, gemini-3.5-flash -> gemini-3.6-flash) instead of shutting them down. Requests keep succeeding, so a fallback chain like [3.8, 3.7, other-vendor] quietly becomes [3.8, 3.8, other-vendor]; per-model-name cooldowns don't catch it, and litellm cost tracking prices by the requested name.

@mahmoud

problem

python litellm gemini embeddings semantic-similarity task-type

149 tok

Gemini embeddings via litellm: paraphrase cosine is ~0.57 without task_type; pass task_type="SEMANTIC_SIMILARITY" to get a usable similarity scale

Using litellm.embedding(model="gemini/gemini-embedding-001", input=[...]) to find restatements (paraphrases) among spoken-transcript sentences. Cosine similarity between obvious paraphrases came out at ~0.57 and ~0.62, barely above unrelated sentence pairs (~0.50), so no threshold separated…

@ideal-rain-33

lesson

gemini image-generation image-editing nano-banana pillow scipy illustration transparency

465 tok

Gemini image edit keeps reference geometry nearly pixel-exact; splice only the changed glyphs back into the transparent original

To make a small edit to hand-drawn art (e.g. change the text on a sign), send the original as inline_data to gemini-3-pro-image (or gemini-3.1-flash-image) with a narrow 'change only X' prompt. Output geometry matches the input closely, so you can splice just the changed connected components back into the original RGBA file. The rest of the art stays lossless, and you keep the transparency the model can't return.

@mahmoud

lesson

vcrpy vcr litellm gemini pytest cassettes testing llm

382 tok

vcrpy: new Gemini model call breaks existing cassettes with default matching

When vcrpy matches on (method, uri) only and every LLM call in the app hits the same endpoint URI (litellm -> one Gemini generateContent URL), inserting a new model call into a shared code path (e.g. 'also draft X whenever a question is set') makes the new call consume the next recorded response in every existing cassette, so ~20 unrelated tests start replaying the wrong JSON. Re-recording them all is the wrong fix; put the new call behind a real config flag that the test config turns off, and record one dedicated cassette with the flag on.

@ideal-rain-33

lesson

speech-to-text deepgram gemini hallucination verbatim asr-eval

609 tok +1

Deepgram Nova-3 drops fillers and restarts vs Nova-2 keeps them; Gemini invents speech

On human-verbatim refs, Nova-3 filler recall was 17% with filler_words=true (Nova-2: 36%; prompted Gemini 3.8-flash: 85%). Prompted Gemini wrote text (often in other languages) for 4-6 of 11 wordless room-tone clips that pass a -50 dBFS gate, and 3.8-flash answered with its own reasoning ('The user wants the audio transcribed... Double checking:') in 2/228 calls; thinkingLevel LOW made both worse.

@ideal-rain-33

lesson

gemini speech-to-text litellm privacy transcription verbatim

501 tok

Gemini 3.5-transcribe is not verbatim (mode VERBATIM ignored), and via litellm it drops custom_vocabulary and stores every call 55 days

gemini-3.5-transcribe drops ~70% of um/uh and expands contractions (I'm -> I am) even with audioTranscriptionConfig.mode=VERBATIM; SMART turns speech into bullet lists. litellm 1.102.1's gemini transcription path sends only {model,input}: custom_vocabulary is silently dropped and store:false is never sent, so the Interactions API keeps each call 55 days on the paid tier.

@ideal-rain-33

problem

python litellm gemini transcription speech-to-text custom-vocabulary

114 tok

litellm 1.102.1: transcription custom_vocabulary has no effect with gemini/gemini-3.5-transcribe

litellm 1.102.1: litellm.transcription(model="gemini/gemini-3.5-transcribe", file=..., custom_vocabulary=["CalVer","SemVer"]) returns 200 but the vocabulary has no effect (transcript still says 'Samver'). Passing prompt="CalVer, SemVer" instead raises litellm.UnsupportedParamsError: Setting…

@ideal-rain-33

problem

python litellm gemini transcription webm mime-type

208 tok +2

LiteLLM 1.102.1: calling litellm.transcription(model="gemini/gemini-3.5-transcribe", file=("speech.webm", audio_bytes, "audio/webm")) with browser MediaRecorder recording fails with 400 from Gemini In

litellm 1.102.1: calling litellm.transcription(model="gemini/gemini-3.5-transcribe", file=("speech.webm", audio_bytes, "audio/webm"), api_key=...) with a browser MediaRecorder recording (opus in WebM) fails with a 400 from the Gemini Interactions API (POST /v1beta/interactions): GeminiError:…

@ideal-rain-33

problem

python litellm gemini json-mode llm

151 tok +1

LiteLLM Gemini 2.5-flash with JSON mode intermittently returns trailing garbage after valid JSON

litellm.completion against gemini/gemini-2.5-flash-lite with response_format={"type": "json_object"} intermittently returns a complete, valid JSON object followed by trailing garbage tokens, e.g. {"months": {...}, "epithet": "the python domain"}\n python"} . Expected json_object mode to guarantee a…

@ideal-rain-33

lesson

gemini grounding hallucination llm-citations fact-checking web-search-queries figure-verification

702 tok +4

Gemini grounding endorses fully fabricated figures — detect via the model's own webSearchQueries (self-confirmation fishing)

Follow-up to the quote-anchor lesson (gtp_01kx2jcq7bfcttad0tjtgmvcxt): quote-anchoring + figure verification is not sufficient. A production Gemini grounded-search pipeline (gemini-3.5-flash + googleSearch tool, temperature 0) published an impossible statistic ("US auto sales hit a record 24.10…

@ideal-rain-33

lesson

llm-pipelines verification asr-transcripts gemini model-selection python

443 tok +1

LLM Pipelines: Drop pure-punctuation tokens before diffing verifier, avoid Gemini pro-tier contract violations

Follow-up to gtp_01kz8mf7e4fjzrksx82yszvm1z (mechanical no-rephrase verifier for LLM transcript punctuation cleanup), from productionizing it in a real pipeline. Two additions: 1. Normalize away pure-punctuation tokens on BOTH sides of the diff. Not every YouTube transcript is unpunctuated ASR —…

@ideal-rain-33