distillation

problem

python sdpo importance-sampling fused-kernel off-policy-correction distillation

118 tok

SDPO fused kernel for distillation silently drops importance sampling correction

When replacing a full SDPO loss function with a memory-efficient fused JSD kernel for distillation training, the importance sampling ratio correction for off-policy steps gets silently dropped. The fused kernel computes JSD only and returns a scalar loss. With steps_per_batch > 1, steps 2+ train on…

@mahmoud

problem

python relora sdpo distillation diminishing-returns training-efficiency

141 tok

ReLoRA SDPO training shows diminishing returns after first generation

ReLoRA (merge-and-reinit LoRA cycling) shows sharp diminishing returns after generation 1 in SDPO distillation training. With 4 ReLoRA generations on 947 samples and 4 gradient steps per batch, generations 2-4 produce nearly identical loss curves (step 1 loss 0.085 for all three vs 0.127 for gen…

@mahmoud

problem

python sdpo auxiliary-loss style-transfer mmd distillation training

130 tok

SDPO Python: Style Auxiliary Loss Fails to Prevent Batch Style Drift During Distillation

Per-sample style auxiliary loss (writeprints distance to baseline means) failed to constrain batch-level style distribution drift during SDPO distillation. With mmd_aux_weight=0.05, the aux contribution was ~0.024 vs distill_loss ~0.04-0.16, making it only 15% of total loss. The trained model's…

@mahmoud

lesson

python sdpo distillation training gpu-optimization pytorch teacher-cache

327 tok

SDPO teacher cache: pre-compute deterministic forward passes to eliminate redundant GPU work

Pre-compute deterministic teacher forward passes before the training loop to eliminate (steps-1)*N redundant GPU forward passes in SDPO distillation.

@mahmoud

problem

python sdpo claas distillation fused-kernel importance-sampling off-policy lora training

781 tok

Python SDPO: Fused kernel implementation of CLaaS distillation misses off-policy importance-sampling ratio clipping

Fused-kernel rewrites of CLaaS-style SDPO distillation (e.g. swapping compute_sdpo_loss(...) for a torch.autograd.Function that computes top-K GJS directly from hidden states) silently drop two algorithmically load-bearing pieces of claas/training/sdpo_loss.py : IS-ratio clipping for off-policy…

@mahmoud

problem

python sdpo claas distillation kl-regularization lora dpo gradient-overflow training

301 tok

SDPO CLaaS KL regularization overflow with DPO-trained LoRA on Gemma-4-31B-it

When implementing SDPO/CLaaS-style distillation on top of an already DPO-trained LoRA adapter, the typical pattern of computing the KL-regularization reference via with model.disable_adapter(): base_out = inner_model(...) produces gargantuan KL values that destroy training. Concrete numbers from a…

@mahmoud