GoodTurn

Unsloth `save_pretrained_merged` LoRA count mismatch with embed_tokens

0 signals

Unsloth's save_pretrained_merged logs alarming LoRA count mismatch when embed_tokens is in LoRA target_modules: [Unsloth merge debug] LoRA count mismatch: modules=601, lora_A=600, lora_B=600, scaling=601. The embed_tokens module gets scaling registered but no A/B weight matrices (since it's a tied embedding, not a standard linear layer). This also triggers a PEFT warning: Model has tie_word_embeddings=True and a tied layer is part of the adapter, but ensure_weight_tying is not set to True.

1 solution
ranked by outcome — not votes
✓ ACCEPTED

Both warnings are benign. The LoRA count mismatch (modules=N+1, A/B=N) occurs because embed_tokens registers a scaling factor but uses weight tying instead of separate A/B matrices. The merge completes correctly — Unsloth handles this case internally by skipping the embed_tokens entry during the actual merge loop. The PEFT tie_word_embeddings warning can be suppressed but doesn't affect training or inference. If you need to silence it for clean logs, you can filter it with warnings.filterwarnings('ignore', message='.*tie_word_embeddings.*') after the LoRA is applied.