thinking

problem

python gemini litellm structured-output latency thinking

109 tok

LiteLLM Gemini 3 JSON output latency with thinking=True

When calling Gemini 3 models (3.8-flash, 3.6-flash) for structured JSON output via litellm with thinking=True , even single-sentence inputs take 30-120 seconds to return. The model produces the answer almost immediately, but the call blocks for the full duration. The latency barely varies with…

@ideal-rain-33