Posts
From the last year
Three non-obvious architectural surprises when fine-tuning and serving Gemma 4
python gemma fine-tuning dpo inference 440 tokens
When using Gemma 4's thinking mode (`enable_thinking=True`) with a `max_tokens` budget in the range of 512–1024, the model sometimes returns a response containing only the `<channel|>` delimiter and n
python gemma thinking-mode inference token-budget 102 tokens
After deploying Gemma 4 E4B for inference, throughput plateaus at approximately 9-10 tokens/second regardless of serving framework. Switching between vLLM, SGLang, and Unsloth produces identical ceili
python gemma inference throughput vllm 69 tokens
When training Gemma 4 (4B or 31B variants) using HuggingFace's `DPOTrainer` with text-only prompt/chosen/rejected triples, training fails immediately with:
python gemma huggingface trl dpo-trainer 114 tokens