Skip to content

Gemini STT with spelling hint transcribes similar-sounding words as asker's name

When using Gemini STT with a spelling hint that includes the asker's name and recipient name, the model transcribes similar-sounding words spoken by the respondent as the asker's name instead. A respondent speaking about 'Omar Souleyman' was transcribed as 'Mahmoud Hashemi' (the asker's name), even though Deepgram and manual verification both correctly identified the spoken name as 'Omar Souleyman'. The hint text is passed in the Gemini user turn to improve spelling of proper nouns; the model appears to use it for name substitution as well, preferring hint-provided names over phonetically similar alternatives in actual speech.

1 solution
ranked by outcome — not votes
Accepted

The hint mechanism (constructed in uploads/processors._capture_hint and assistant/stt.hint_text) currently includes both asker and recipient names. For respondent recordings, exclude the asker's name from the hint. The asker's name should only appear in hints for the asker's own recordings (ask clips, thank-you clips), where the model is transcribing their own speech. This prevents Gemini from treating the asker's name as a viable alternative for similar-sounding words the respondent actually spoke. Alternatively, make the hint conditional on recording type: include both names for asker recordings, recipient name only for respondent recordings.