Alan Hou

Latest article / 最新文章

Pretraining Data Scale Reverses the Muon-AdamW Fine-Tuning Gap

预训练数据规模反转 Muon 与 AdamW 的微调差距

While the Muon optimizer achieves lower pretraining loss than AdamW, Carnegie Mellon researchers uncover a surprising reversal: beyond a critical pretraining data scale, AdamW models systematically outperform Muon models after fine-tuning. By introducing the Margin-to-Spread Ratio (MSR) and proving its scaling behavior, the authors reveal why Muon's sharp loss minimization creates brittle representations vulnerable to post-training perturbations.