MiMo V2 Pro Open-Source Teased

🔑 Enhanced Key Takeaways

• •MiMo-V2-Flash, released by Xiaomi in early 2025, features a 309B total parameter MoE architecture with only 15B active parameters for high efficiency[1][3].
• •The model employs Hybrid Sliding Window Attention to reduce KV cache by 6x and Multi-Token Prediction for 3x inference speed boost[1].
• •Xiaomi claims MiMo-V2-Flash outperforms open-source competitors on SWE-Bench Verified (73.4%) and matches Claude 4.5 Sonnet on coding at lower cost[4].
• •Trained on 27 trillion tokens with FP8 precision and post-trained using Multi-Teacher On-Policy Distillation and large-scale agentic RL[1][3].

🛠️ Technical Deep Dive

• •Architecture: Mixture-of-Experts (MoE) with 309B total parameters, 15B active; Hybrid Sliding Window Attention (SWA) reduces KV cache by 6x[1][3][6].
• •Multi-Token Prediction (MTP): Native integration with 0.33B params per block using dense FFN (not MoE), triples inference speed, aids RL training[1].
• •Training: Pre-trained on 27T tokens (FP8, native 32K extended to 256K context); SFT on millions of samples; MOPD distillation; 100K+ GitHub agent tasks; multimodal verifier[1][3].
• •Performance: AIME 2025 math 94.1%; excels multilingual coding, long-context (LongBench V2, MRCR); concerns over benchmark inconsistencies[1][3].

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline