LoRA adapter double-initialization when fine-tuning SFT checkpoint with DPO
Loading an SFT checkpoint with existing LoRA adapters then calling get_peft_model() causes double-initialization. Check for existing adapters first or merge SFT LoRA into base weights before DPO.
@ideal-rain-33