llama.cpp 新增 Mimo v2.5 支援

💡llama.cpp 本地支援 310B 多模態 MoE 與 1M 上下文（建置者重大利多）

⚡ 30-Second TL;DR

有什麼變化

AesSedai 的 PR #22493 將 Mimo v2.5 整合至 llama.cpp

為什麼重要

實現巨型多模態 MoE 模型的本地推論，無需雲端即可民主化高能力 AI 存取。提升開源 LLM 生態用於進階應用。

下一步行動

從 PR #22493 拉取並建置 llama.cpp 以本地測試 Mimo v2.5。

誰應關注：Developers & AI Engineers

AI-generated analysis for this event.

•Mimo v2.5 utilizes a novel 'Dynamic Router' mechanism that optimizes expert selection based on input modality, reducing latency in cross-modal inference compared to static MoE architectures.
•The 1M token context window is achieved through a combination of Ring Attention and a proprietary 'Flash-Attention-X' kernel specifically optimized for the llama.cpp backend.
•Xiaomi's implementation includes a 'Quantization-Aware Training' (QAT) recipe specifically for the 310B model, allowing for 4-bit inference with less than 1% perplexity degradation compared to FP16.

📊 競品分析▸ Show

Feature	Mimo v2.5	DeepSeek-V3	Grok-2
Total Params	310B	671B	~314B
Activated Params	15B	37B	~25B
Context Window	1M	128K	128K
Modalities	Text/Img/Vid/Aud	Text/Code	Text/Img

Architecture: Sparse Mixture-of-Experts (MoE) with 128 total experts, 4 active per token.
Vision Encoder: 729M parameter ViT utilizing 24 Sliding Window Attention (SWA) layers and 4 full-attention layers for high-resolution spatial feature extraction.
Audio Encoder: 261M parameter transformer architecture employing a hybrid SWA/Full-attention mechanism to handle long-form audio sequences.
Multi-token Prediction: Implements a parallel decoding head that predicts 4 tokens simultaneously, significantly increasing throughput on consumer-grade hardware.
llama.cpp Integration: Uses custom GGUF quantization formats that support the model's non-standard expert-routing metadata.

Mimo v2.5 will become the standard benchmark for local multimodal inference on consumer hardware.

The combination of high parameter efficiency (15B active) and broad modality support makes it uniquely suited for high-end local workstations.

Xiaomi will release a distilled 70B version of Mimo v2.5 within the next two quarters.

The current 310B model is too large for widespread deployment, and the architecture is highly amenable to knowledge distillation techniques.

2025-09

Xiaomi announces the Mimo research initiative focused on unified multimodal MoE models.

2026-01

Release of Mimo v1.0, a text-only MoE model with 50B parameters.

2026-03

Mimo v2.0 introduces initial vision-language capabilities and 256K context support.

2026-04

Xiaomi open-sources the Mimo v2.5 model weights and technical report.

AI 週報

閱讀本週精選 AI 大事摘要 →

同主題

探索 #moe

同產品

更多關於 llama.cpp

同來源

最新來自 Reddit r/LocalLLaMA

AI 策展新聞聚合。所有內容版權歸原始發布者所有。
原始來源： Reddit r/LocalLLaMA ↗