Xiaomi MiMo UltraSpeed: 1000+ tokens/s inference breakthrough

🔑 Enhanced Key Takeaways

•MiMo-V2.5-Pro is a 1.02 trillion-parameter Mixture-of-Experts (MoE) model with 42 billion active parameters, utilizing a hybrid attention architecture that includes Local Sliding Window Attention (SWA) and Global Attention (GA) to achieve a 1M-token context window and reduce KV-cache storage by nearly 7x.
•The UltraSpeed mode is specifically enabled by an FP4-quantized backbone, which applies MXFP4 quantization exclusively to the MoE experts while retaining original precision for other model modules, thereby reducing model size and optimizing hardware bandwidth utilization.
•TileRT's inference engine, a key component of the UltraSpeed breakthrough, optimizes GPU operation scheduling by employing a persistent Engine Kernel that resides on the GPU, significantly reducing kernel startup overheads and memory read/write delays compared to traditional inference systems.
•The MiMo-V2.5-Pro model integrates Multi-Token Prediction (MTP) modules with dense Feed-Forward Networks (FFNs) natively for both training and inference, which is reported to roughly triple output throughput and accelerate reinforcement learning rollouts.
•The API for MiMo-V2.5-Pro UltraSpeed is offered at a promotional price, costing three times the standard MiMo-V2.5-Pro API but delivering approximately ten times the generation speed, indicating a significant performance-to-cost improvement for high-speed applications.

📊 Competitor Analysis▸ Show

Feature/Model	Xiaomi MiMo-V2.5-Pro UltraSpeed	DeepSeek V4 Pro	NVIDIA Nemotron 3 Ultra	Step-3 (VLM)	GLM-5.1 High-Speed API
Total Parameters	1.02 Trillion	~1 Trillion	550 Billion	321 Billion	(Part of GLM-5.1 model family)
Active Parameters	42 Billion	~32-37 Billion	55 Billion	38 Billion	N/A
Inference Speed	1000+ tokens/s	N/A (Focus on cost-efficiency)	300+ tokens/s	Up to 4,039 tokens/s (4K context, FP8, no MTP)	400 tokens/s
Context Window	1 Million tokens	1 Million tokens	262k tokens (BF16), 1M (NVFP4)	N/A (Optimized for long-context decoding)	N/A
Key Optimizations	Model-system co-design, Hybrid SWA, MTP, Expert-Only FP4 Quantization	Engram memory architecture, MoE	Hybrid Mamba-Transformer, LatentMoE	Multi-Matrix Factorization Attention (MFA), Attention-FFN Disaggregation (AFD)	Persistent Engine Kernel, specialized GPU nodes
Pricing/Cost	3x MiMo-V2.5-Pro API, 10x speed	~3x cheaper than Gemini 3.1 Pro, ~12x cheaper than GPT-5.5 (comparable intelligence)	30% lower per-task cost (due to token efficiency)	Cost-effective decoding	N/A (Available to select enterprise clients)
Open-Source	MiMo-V2.5-Pro is open-source	Open-source weights expected (Apache 2.0)	Open-weight	Open-sourced on July 31, 2025	N/A

🛠️ Technical Deep Dive

Model Architecture: MiMo-V2.5-Pro is a Mixture-of-Experts (MoE) model with a total of 1.02 trillion parameters and 42 billion active parameters per token.
Attention Mechanism: It employs a hybrid attention architecture, interleaving Local Sliding Window Attention (SWA) and Global Attention (GA) at a 6:1 ratio with a 128-token window. This design significantly reduces KV-cache storage by nearly 7x at long contexts while maintaining performance.
Context Window: The model supports an extended context length of up to 1 million tokens.
Multi-Token Prediction (MTP): A lightweight MTP module with dense FFNs is natively integrated into the architecture for both training and inference, which is crucial for tripling output throughput and accelerating RL rollouts.
Quantization for UltraSpeed: The UltraSpeed mode specifically utilizes an FP4-quantized backbone, applying MXFP4 quantization to the MoE experts while keeping other modules at their original precision to optimize for size and hardware bandwidth without significant accuracy degradation.
Training Process: Pre-training was conducted on 27 trillion tokens using FP8 mixed precision at a native 32K sequence length. Post-training follows a three-stage paradigm: Supervised Fine-Tuning (SFT), Domain-Specialized Training (using RL across various domains), and Multi-Teacher On-Policy Distillation (MOPD).
TileRT Inference Engine: The underlying TileRT engine optimizes GPU operation scheduling by restructuring it into a persistent Engine Kernel that resides on the GPU, reducing kernel startup and memory read/write delays. It also pushes communication directly into the execution pipeline for multi-card scenarios, specializing GPU nodes into functional Workers.

🔮 Future ImplicationsAI analysis grounded in cited sources

Accelerated adoption of complex AI agents in real-time applications.

Achieving 1000+ tokens/s for trillion-parameter models removes a significant latency bottleneck, making sophisticated agentic workflows and real-time AI interactions more practical and widespread.

Intensified competition in the high-performance LLM inference market.

Xiaomi's breakthrough in inference speed for open-source, frontier-tier models will pressure other AI developers and cloud providers to innovate further in efficiency and speed to remain competitive.

Model-system co-design will become a standard for LLM optimization.

The demonstrated performance gains from jointly optimizing model architecture and inference systems, as seen with UltraSpeed, will likely establish co-design as a critical methodology for future LLM deployments on general-purpose hardware.

⏳ Timeline

2025-04-30

Xiaomi released its first open-source large language model, MiMo-7B.

2025-12-17

Xiaomi launched MiMo-V2-Flash, a 309 billion parameter model.

2026-03-18

Xiaomi publicly introduced the MiMo-V2 series, including MiMo-V2-Pro (1 trillion parameters), MiMo-V2-Omni, and MiMo-V2-TTS.

2026-04-22

Xiaomi released and open-sourced MiMo-V2.5-Pro (1.02T total, 42B active parameters) and MiMo-V2.5.

2026-05-21

TileRT's high-performance inference engine, a partner in the UltraSpeed development, was noted to be in production, powering GLM-5.1 on Z.ai MaaS.

2026-06-08

Xiaomi and TileRT officially launched MiMo-V2.5-Pro UltraSpeed, achieving over 1000 tokens/s inference for trillion-parameter models.

Xiaomi MiMo UltraSpeed: 1000+ tokens/s inference breakthrough

⚡ 30-Second TL;DR

Key Points

🧠 Deep Insight

🔑 Enhanced Key Takeaways

🛠️ Technical Deep Dive

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline

📎 Sources (22)

👉Related Updates