LiteLLM Gemini 2.5-flash with JSON mode intermittently returns trailing garbage after valid JSON
litellm.completion against gemini/gemini-2.5-flash-lite with response_format={"type": "json_object"} intermittently returns a complete, valid JSON object followed by trailing garbage tokens, e.g. {"months": {...}, "epithet": "the python domain"}\n python"}. Expected json_object mode to guarantee a parseable body. json.loads raises JSONDecodeError('Extra data') on the whole string, so a fail-open wrapper silently discarded every response and users only ever saw the deterministic fallback path — no error surfaced anywhere. Retrying doesn't help reliably; the junk suffix appears often enough at max_tokens=1200 with a JSON-heavy prompt that the LLM feature effectively never ships output.
gemini-2.5-flash-lite's JSON mode constrains the decode enough to produce a valid object but does not reliably stop after the closing brace; a few junk tokens can follow. Parse the FIRST JSON value and ignore the trailer instead of parsing the whole string:
content = response.choices[0].message.content
data, _end = json.JSONDecoder().raw_decode(content.strip())raw_decode returns (obj, end_index) after the first complete JSON value and never looks at the rest, so the trailing junk is harmless. This is strictly safer than json.loads for any fail-open LLM JSON path: a well-formed object plus garbage should not count as a failure.
Diagnosis tip: if your fail-open wrapper swallows exceptions, temporarily monkeypatch litellm.completion with a spy that captures choices[0].message.content — the junk suffix is invisible in the parsed-or-fallback output and only shows up in the raw string.