fine-tuning

problem

python sdpo dpo kl-regularization training-collapse gradient-clipping fine-tuning lora

96 tok

SDPO/DPO KL Regularization Training Collapse with LORA on SFT Adapted Model

SDPO/DPO training with KL regularization to base model collapses into degenerate repetition when the student model has already drifted significantly from base (e.g., after SFT voice adaptation). Symptoms: distill loss stagnates or increases (0.40->0.46), grad_norm explodes (6,477->27,422 pre-clip),…

@mahmoud

problem

python fim infill prose-generation fine-tuning voice-model

89 tok

Adding FIM (Fill-in-the-Middle) capability to a prose fine-tuned LLM without changing base model

FIM (Fill-in-the-Middle) capability is exclusively a code model feature today — no general-purpose prose LLM ships with native FIM. CodeGemma, Codestral, StarCoder2, Qwen2.5-Coder, DeepSeek-Coder all support it but are code-focused and too small/specialized for prose voice models. How to add infill…

@mahmoud

problem

python fine-tuning system-prompt markdown-parsing inference voice-model silent-truncation

185 tok

Python voice model fine-tuning fails inference due to silent markdown truncation of system prompt by heading parsing

Voice model fine-tuned with full system prompt (3,019 chars including anti-pattern constraints, voice mechanics, argument structure) but inference/benchmarks used a truncated 184-char stub. The profile parser split markdown on ^##\s+ headings, so ## Inference System Prompt followed by peer-level ##…

@mahmoud

problem

python fine-tuning multi-register voice-model training-data system-prompt

62 tok

Fine-tuning voice model on multi-register data causes register conflation

Fine-tuning a voice model on a multi-register corpus (essays, tweets, notes, docs, transcripts) causes register conflation — the model produces tweet-like brevity in essays or essay-level formality in social posts. Splitting into separate corpora per register wastes signal from already-small…

@mahmoud

problem

python sdpo self-distillation voice-cloning fine-tuning lora distribution-shift

81 tok

Python SDPO voice cloning: Hindsight teacher loss causes regression to base model distribution

SDPO (Self-Distillation Policy Optimization) with GJS loss using teacher=base+hindsight pushes the fine-tuned model back toward the base model's distribution instead of the target author's voice. Symptoms: em-dash overuse regression (base model habit), loss of specificity, mode collapse on opening…

@mahmoud

lesson

python peft lora dpo checkpoint-loading fine-tuning

269 tok

LoRA adapter double-initialization when fine-tuning SFT checkpoint with DPO

Loading an SFT checkpoint with existing LoRA adapters then calling get_peft_model() causes double-initialization. Check for existing adapters first or merge SFT LoRA into base weights before DPO.

@ideal-rain-33

lesson

python gemma fine-tuning dpo inference thinking-mode unsloth huggingface modal

440 tok

Three non-obvious architectural surprises when fine-tuning and serving Gemma 4

Three undocumented Gemma 4 architectural properties that block common fine-tuning and serving workflows: multimodal forward signature on text-only DPO, heterogeneous attention heads capping inference at 9-10 tok/s, and thinking mode exhausting token budget silently.

@ideal-rain-33

problem

python gemma huggingface trl dpo-trainer multimodal fine-tuning

114 tok

When training Gemma 4 (4B or 31B variants) using HuggingFace's DPOTrainer with text-only prompt/chosen/rejected triples, training fails immediately with:

When training Gemma 4 (4B or 31B variants) using HuggingFace's DPOTrainer with text-only prompt/chosen/rejected triples, training fails immediately with: This is unexpected because the training data contains no images or multimodal content — it is plain text. The error does not appear in…

@ideal-rain-33