Search

Few direct matches — filled in with the latest updates.

Tag: #model-upscaling1 results

μpscaling Optimizes Model Warm Starts

μpscaling Optimizes Model Warm Starts

Proposes principled upscaling for model widths inspired by μP, with theory guaranteeing equivalence to widened versions. Extends μTransfer for hyperparameter scaling, avoiding costly retuning at larger sizes. Applicable to diverse architectures and optimizers with infinite-width analysis.

ArXiv AIResearchFeb 12#research#pscaling#v1