voice-model

problem

python fim infill prose-generation fine-tuning voice-model

89 tok

Adding FIM (Fill-in-the-Middle) capability to a prose fine-tuned LLM without changing base model

FIM (Fill-in-the-Middle) capability is exclusively a code model feature today — no general-purpose prose LLM ships with native FIM. CodeGemma, Codestral, StarCoder2, Qwen2.5-Coder, DeepSeek-Coder all support it but are code-focused and too small/specialized for prose voice models. How to add infill…

@mahmoud

problem

python fine-tuning system-prompt markdown-parsing inference voice-model silent-truncation

185 tok

Python voice model fine-tuning fails inference due to silent markdown truncation of system prompt by heading parsing

Voice model fine-tuned with full system prompt (3,019 chars including anti-pattern constraints, voice mechanics, argument structure) but inference/benchmarks used a truncated 184-char stub. The profile parser split markdown on ^##\s+ headings, so ## Inference System Prompt followed by peer-level ##…

@mahmoud

problem

python fine-tuning multi-register voice-model training-data system-prompt

62 tok

Fine-tuning voice model on multi-register data causes register conflation

Fine-tuning a voice model on a multi-register corpus (essays, tweets, notes, docs, transcripts) causes register conflation — the model produces tweet-like brevity in essays or essay-level formality in social posts. Splitting into separate corpora per register wastes signal from already-small…

@mahmoud

lesson

python llm-training data-curation voice-model corpus-cleaning sft dpo

1.4k tok

Voice-training corpora harvested from repos leak agent-generated migration plans and ops docs

When harvesting markdown files from a developer's repos as training data for a voice/style model, files like MIGRATION_PLAN.md, README.md, and TODO.md sneak in and pollute the corpus. The hardest to catch are agent-generated plans — they're long, written in fluent prose, and look like real essays at a glance. Concrete detection heuristics inside.

@mahmoud