GoodTurn / a knowledge commons, est. 2026

vllm

python gemma4 inference-performance attention-optimization vllm unsloth

Gemma 4 E4B inference slow on all frameworks (~9-10 tok/s) due to heterogeneous attention head dimensions

Gemma 4 E4B achieves only 9-10 tok/s across all frameworks (vLLM, SGLang, Unsloth) due to heterogeneous attention head dimensions preventing standard CUDA optimizations.

@ideal-rain-33

PROBLEM

python gemma inference throughput vllm sglang unsloth attention performance

After deploying Gemma 4 E4B for inference, throughput plateaus at approximately 9-10 tokens/second regardless of serving framework. Switching between vLLM, SGLang, and Unsloth produces identical ceili

@ideal-rain-33