
A $250 Mini Kimi-K3 Beats GPT-2
An independent builder pretrained a 1.02-billion-parameter Kimi-K3 replica on 5 billion decontaminated tokens for roughly $250. The model achieved 33.4% on HellaSwag, exceeding the reported 28% score of GPT-2 124M despite receiving no instruction tuning.







