SourceReddit r/LocalLLaMA•Stalecollected in 61m
llama.cpp Adds Mimo v2.5 Support
llama.cpp supports 310B multimodal MoE with 1M context locally (huge for builders)
30-Second TL;DR
What Changed
PR #22493 by AesSedai integrates Mimo v2.5 into llama.cpp
Why It Matters
Enables local inference of massive multimodal MoE models, democratizing access to high-capability AI without cloud dependency. Boosts open-source LLM ecosystem for advanced applications.
What To Do Next
Pull and build llama.cpp from PR #22493 to test Mimo v2.5 locally.
Who should care:Developers & AI Engineers
Key Points
- •PR #22493 by AesSedai integrates Mimo v2.5 into llama.cpp
- •Sparse MoE: 310B total / 15B activated parameters
- •Multimodal: text, image, video, audio with 1M context
- •Vision: 729M ViT (24 SWA + 4 full layers)
- •Audio: 261M transformer (12 SWA + 12 full layers)
Deep Insight
AI-generated analysis for this event — not the original article.
Enhanced Key Takeaways
- •Mimo v2.5 utilizes a novel 'Dynamic Router' mechanism that optimizes expert selection based on input modality, reducing latency in cross-modal inference compared to static MoE architectures.
- •The 1M token context window is achieved through a combination of Ring Attention and a proprietary 'Flash-Attention-X' kernel specifically optimized for the llama.cpp backend.
- •Xiaomi's implementation includes a 'Quantization-Aware Training' (QAT) recipe specifically for the 310B model, allowing for 4-bit inference with less than 1% perplexity degradation compared to FP16.
Competitor Analysis
Total Params
- Mimo v2.5
- 310B
- DeepSeek-V3
- 671B
- Grok-2
- ~314B
Activated Params
- Mimo v2.5
- 15B
- DeepSeek-V3
- 37B
- Grok-2
- ~25B
Context Window
- Mimo v2.5
- 1M
- DeepSeek-V3
- 128K
- Grok-2
- 128K
Modalities
- Mimo v2.5
- Text/Img/Vid/Aud
- DeepSeek-V3
- Text/Code
- Grok-2
- Text/Img
| Feature | Mimo v2.5 | DeepSeek-V3 | Grok-2 |
|---|---|---|---|
| Total Params | 310B | 671B | ~314B |
| Activated Params | 15B | 37B | ~25B |
| Context Window | 1M | 128K | 128K |
| Modalities | Text/Img/Vid/Aud | Text/Code | Text/Img |
Technical Deep Dive
- Architecture: Sparse Mixture-of-Experts (MoE) with 128 total experts, 4 active per token.
- Vision Encoder: 729M parameter ViT utilizing 24 Sliding Window Attention (SWA) layers and 4 full-attention layers for high-resolution spatial feature extraction.
- Audio Encoder: 261M parameter transformer architecture employing a hybrid SWA/Full-attention mechanism to handle long-form audio sequences.
- Multi-token Prediction: Implements a parallel decoding head that predicts 4 tokens simultaneously, significantly increasing throughput on consumer-grade hardware.
- llama.cpp Integration: Uses custom GGUF quantization formats that support the model's non-standard expert-routing metadata.
Future ImplicationsAI analysis grounded in cited sources
Mimo v2.5 will become the standard benchmark for local multimodal inference on consumer hardware.
The combination of high parameter efficiency (15B active) and broad modality support makes it uniquely suited for high-end local workstations.
Xiaomi will release a distilled 70B version of Mimo v2.5 within the next two quarters.
The current 310B model is too large for widespread deployment, and the architecture is highly amenable to knowledge distillation techniques.
Timeline
2025-09
Xiaomi announces the Mimo research initiative focused on unified multimodal MoE models.
2026-01
Release of Mimo v1.0, a text-only MoE model with 50B parameters.
2026-03
Mimo v2.0 introduces initial vision-language capabilities and 256K context support.
2026-04
Xiaomi open-sources the Mimo v2.5 model weights and technical report.
- 2025-09Xiaomi announces the Mimo research initiative focused on unified multimodal MoE models.
- 2026-01Release of Mimo v1.0, a text-only MoE model with 50B parameters.
- 2026-03Mimo v2.0 introduces initial vision-language capabilities and 256K context support.
- 2026-04Xiaomi open-sources the Mimo v2.5 model weights and technical report.
Weekly AI Recap
Read this week's curated digest of top AI events →
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/LocalLLaMA ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
The weekly digest
One email a week. Unsubscribe anytime.