來源Reddit r/LocalLLaMA•較早收集於 16h
mlx-lm 即將支援 Qwen 3.5 的 MTP
#apple-silicon#mlx#throughputmlx-lmmlx-lmqwen-3.5mtpm4-pro
💡mlx-lm PR 讓 M4 Pro 上 Qwen 3.5 推理快 1.5 倍，透過 MTP (24字)
⚡ 30 秒速覽
有什麼變化
吞吐量增 1.5 倍：15.3 至 23.3 tok/s
為什麼重要
提升 Apple 硬體上 Qwen 模型的生成速度，更適合本地部署。PR 整合可讓 mlx 生態標準化 MTP。
下一步行動
檢視並在 Apple Silicon 上測試 mlx-lm PR #990 的 Qwen 3.5 MTP。
誰應關注：Developers & AI Engineers
關鍵要點
- •吞吐量增 1.5 倍：15.3 至 23.3 tok/s
- •令牌接受率 80.6%
- •M4 Pro 上 Qwen3.5-27B 4-bit
- •AirRunner 的 PR #990，早期支援上線
- •針對 Apple Silicon 本地推理
🧠 深度解析
本篇為 AI 生成分析，非原文內容。
🔑 增強重點摘要
- •Multi-Token Prediction (MTP) in mlx-lm leverages a speculative decoding-like architecture where the model predicts multiple future tokens simultaneously, reducing the number of sequential forward passes required during inference.
- •The implementation specifically optimizes for Apple Silicon's Unified Memory Architecture (UMA), utilizing custom Metal kernels to minimize latency overhead when processing the auxiliary MTP heads.
- •The 80.6% acceptance rate indicates high confidence in the auxiliary heads, suggesting that the Qwen 3.5 architecture was specifically fine-tuned or trained with MTP objectives to align with the primary model's distribution.
📊 競品分析▸ Show
| Feature | mlx-lm (MTP) | llama.cpp (Speculative) | vLLM (Speculative) |
|---|---|---|---|
| Hardware Target | Apple Silicon (Metal) | CPU/GPU (General) | GPU (NVIDIA/AMD) |
| Decoding Method | MTP (Native) | Speculative Decoding | Speculative Decoding |
| Throughput Gain | ~1.5x (M4 Pro) | Variable (Draft Model) | Variable (Draft Model) |
| Ease of Use | High (Apple-native) | Moderate | High (Server-side) |
🛠️ 技術深入
- MTP Architecture: Utilizes additional output heads trained to predict the next N tokens in a single forward pass, rather than relying on a separate, smaller draft model.
- Memory Efficiency: By avoiding a separate draft model, the system maintains a lower memory footprint, which is critical for running 27B parameter models on consumer-grade Apple Silicon.
- Kernel Optimization: The PR utilizes specialized Metal Performance Shaders (MPS) to parallelize the verification of multiple tokens, significantly reducing the bottleneck typically associated with autoregressive generation.
- Quantization Compatibility: The implementation is optimized for 4-bit quantized weights (likely using Q4_0 or similar formats supported by mlx), ensuring that the MTP heads remain performant despite weight compression.
🔮 前景展望基於引用來源的 AI 分析
MTP will become the default inference mode for local LLMs on Apple Silicon.
The significant throughput gains without the memory overhead of traditional speculative decoding make it the most efficient path for high-parameter models on constrained hardware.
Future Qwen iterations will integrate MTP heads into the base training objective.
The high acceptance rate observed in the Qwen 3.5 implementation suggests that native MTP training is superior to post-hoc head attachment.
⏳ 時間線
2024-01
MLX framework released by Apple Machine Learning Research.
2025-09
Qwen 3.5 series announced with support for advanced architectural optimizations.
2026-03
PR #990 merged into mlx-lm, enabling MTP for Qwen 3.5.
📰
AI 週報
閱讀本週精選 AI 大事摘要 →
👉相關動態
AI 策展新聞聚合。所有內容版權歸原始發布者所有。
原始來源： Reddit r/LocalLLaMA ↗
每週電子報
每週一封，可隨時退訂。