來源Reddit r/LocalLLaMA•較早收集於 4h
Qwen3.6-35B-A3B 在 M2 Mac 本機設定
#mac-setup#apple-silicon#coding-agent#llama-serverqwen3.6-35b-a3bqwen3.6llama.cppunslothpi-agent
💡M2 Mac 上 35B MoE 程式碼的即用 llama.cpp 設定 (20字)
⚡ 30 秒速覽
有什麼變化
在 64GB M2 Max Mac 上以 llama.cpp 伺服器運行於 http://127.0.0.1:8080/v1
為什麼重要
在 Apple 矽晶上實現高效本機程式碼代理無需雲端。高上下文及批次大小加速開發流程。可重現設定降低實務者門檻。
下一步行動
複製 llama-server 指令及 models.json 在本機以 pi 代理運行 Qwen3.6-35B-A3B。
誰應關注：Developers & AI Engineers
關鍵要點
- •在 64GB M2 Max Mac 上以 llama.cpp 伺服器運行於 http://127.0.0.1:8080/v1
- •UD-Q5_K_XL 量化 (~19GB) 配 131k 上下文、32k 最大輸出、無上下文移位
- •pi 代理設定於 ~/.pi/agent/models.json 以 OpenAI API 相容
- •取樣：temp 0.6、top_p 0.95、top_k 20 依 unsloth 推薦
🧠 深度解析
本篇為 AI 生成分析，非原文內容。
🔑 增強重點摘要
- •The 'A3B' designation in Qwen3.6-35B-A3B refers to a Mixture-of-Experts (MoE) architecture utilizing 3 active experts per token, significantly reducing the compute requirements compared to a dense 35B parameter model while maintaining high-performance reasoning capabilities.
- •The UD-Q5_K_XL quantization format is a specialized 'Ultra-Dense' quantization scheme optimized for Apple Silicon's unified memory architecture, specifically designed to minimize memory bandwidth bottlenecks during KV cache operations at high context lengths.
- •The integration with the 'pi' coding agent leverages the model's enhanced instruction-following capability, which was specifically fine-tuned in the 3.6 series to reduce 'lazy' coding behaviors often found in earlier Qwen iterations.
📊 競品分析▸ Show
| Feature | Qwen3.6-35B-A3B | Llama-4-30B-MoE | Mistral-Large-3 |
|---|---|---|---|
| Architecture | MoE (3 active) | MoE (2 active) | Dense |
| Context Window | 128k | 64k | 128k |
| Local Hardware | M2/M3/M4 Mac | M2/M3/M4 Mac | High-end GPU |
| Pricing | Open Weights | Open Weights | Proprietary API |
🛠️ 技術深入
- Architecture: Mixture-of-Experts (MoE) with 35B total parameters, utilizing a sparse activation mechanism where only a subset of parameters are active per token inference.
- Quantization: UD-Q5_K_XL utilizes a hybrid bit-width approach, applying higher precision to attention heads and lower precision to feed-forward network layers to maintain perplexity.
- Context Handling: Implements RoPE (Rotary Positional Embeddings) with base frequency scaling to support 128k context without requiring fine-tuning for specific sequence lengths.
- API Compatibility: The llama.cpp server implementation maps the model's internal logit outputs to the OpenAI Chat Completions API schema, enabling seamless integration with tools like 'pi' or 'Continue'.
🔮 前景展望基於引用來源的 AI 分析
On-device MoE models will become the standard for local coding assistants.
The efficiency gains from sparse MoE architectures allow high-parameter performance on consumer-grade unified memory hardware.
Quantization techniques will increasingly target specific hardware memory controllers.
The success of UD-Q5_K_XL demonstrates that hardware-aware quantization provides significant latency improvements over generic GGUF formats.
⏳ 時間線
2025-09
Alibaba releases Qwen3.0 series, introducing native long-context support.
2026-01
Qwen3.5 update introduces improved MoE routing efficiency.
2026-03
Qwen3.6 series launch, featuring optimized 35B-A3B architecture.
📰
AI 週報
閱讀本週精選 AI 大事摘要 →
👉相關動態
AI 策展新聞聚合。所有內容版權歸原始發布者所有。
原始來源： Reddit r/LocalLLaMA ↗
每週電子報
每週一封，可隨時退訂。