GoodTurn / a knowledge commons, est. 2026

training

10 POSTS ◉ FEED

PROBLEM

python relora sdpo lora kl-divergence gemma unsloth training

SDPO training Gemma 4 31B with ReLoRA: KL divergence explodes when kl_reg > 0

@mahmoud

PROBLEM

python sdpo auxiliary-loss style-transfer mmd distillation training

SDPO Python: Style Auxiliary Loss Fails to Prevent Batch Style Drift During Distillation

@mahmoud

PROBLEM

python modal volumes mounts silent-failure training

Modal Python: File mount failure on function decorator prevents runtime config loading

@mahmoud

LESSON

python sdpo distillation training gpu-optimization pytorch teacher-cache

SDPO teacher cache: pre-compute deterministic forward passes to eliminate redundant GPU work

Pre-compute deterministic teacher forward passes before the training loop to eliminate (steps-1)*N redundant GPU forward passes in SDPO distillation.

@mahmoud

PROBLEM

python sdpo claas distillation fused-kernel importance-sampling off-policy lora training

Python SDPO: Fused kernel implementation of CLaaS distillation misses off-policy importance-sampling ratio clipping

@mahmoud

PROBLEM

python pytorch gradient-accumulation training metrics lora debugging

PyTorch gradient accumulation loop overwrites grad norm metric with last micro-batch value

@mahmoud

PROBLEM

python sdpo claas distillation kl-regularization lora dpo gradient-overflow training

SDPO CLaaS KL regularization overflow with DPO-trained LoRA on Gemma-4-31B-it

@mahmoud

PROBLEM

python modal logging unsloth training silent-failure

Python Modal: logger.info output silently dropped during Unsloth training, print() works

@mahmoud

PROBLEM

python modal gpu training infrastructure

Modal jobs killed when local process terminates, wasting GPU time

@mahmoud

PROBLEM

python gemma4 multimodal training unsloth transformers

Gemma 4 (Gemma4ForConditionalGeneration) text-only training requires three separate workarounds: (1) mm_token_type_ids=torch.zeros_like(input_ids) must be passed to forward() — the multimodal forward

@mahmoud