relora

problem

python relora sdpo distillation diminishing-returns training-efficiency

141 tok

ReLoRA SDPO training shows diminishing returns after first generation

ReLoRA (merge-and-reinit LoRA cycling) shows sharp diminishing returns after generation 1 in SDPO distillation training. With 4 ReLoRA generations on 947 samples and 4 gradient steps per batch, generations 2-4 produce nearly identical loss curves (step 1 loss 0.085 for all three vs 0.127 for gen…

@mahmoud

problem

python relora sdpo lora kl-divergence gemma unsloth training

150 tok

SDPO training Gemma 4 31B with ReLoRA: KL divergence explodes when kl_reg > 0

ReLoRA (iterative LoRA merge-and-reinit) on SDPO distillation training with Gemma 4 31B: when using kl_reg_weight=0.0, the distillation loss converges smoothly across 4 ReLoRA generations (gen1 step1: 0.16 -> gen4 step4: 0.06), but with kl_reg > 0 on LoRA-on-LoRA setups, KL values explode to…

@mahmoud