๐ฆReddit r/LocalLLaMAโขFreshcollected in 4h
Qwen3.6-35B-A3B Local Setup on M2 Mac
๐กReady-to-run llama.cpp config for 35B MoE coding on M2 Mac
โก 30-Second TL;DR
What Changed
Runs on M2 Max 64GB Mac with llama.cpp server at http://127.0.0.1:8080/v1
Why It Matters
Enables efficient local coding agent on Apple silicon without cloud. High context and batch sizes speed up dev workflows. Reproducible config lowers setup barriers for practitioners.
What To Do Next
Copy the llama-server command and models.json to run Qwen3.6-35B-A3B with pi agent locally.
Who should care:Developers & AI Engineers
๐ง Deep Insight
AI-generated analysis for this event.
๐ Enhanced Key Takeaways
- โขThe 'A3B' designation in Qwen3.6-35B-A3B refers to a Mixture-of-Experts (MoE) architecture utilizing 3 active experts per token, significantly reducing the compute requirements compared to a dense 35B parameter model while maintaining high-performance reasoning capabilities.
- โขThe UD-Q5_K_XL quantization format is a specialized 'Ultra-Dense' quantization scheme optimized for Apple Silicon's unified memory architecture, specifically designed to minimize memory bandwidth bottlenecks during KV cache operations at high context lengths.
- โขThe integration with the 'pi' coding agent leverages the model's enhanced instruction-following capability, which was specifically fine-tuned in the 3.6 series to reduce 'lazy' coding behaviors often found in earlier Qwen iterations.
๐ Competitor Analysisโธ Show
| Feature | Qwen3.6-35B-A3B | Llama-4-30B-MoE | Mistral-Large-3 |
|---|---|---|---|
| Architecture | MoE (3 active) | MoE (2 active) | Dense |
| Context Window | 128k | 64k | 128k |
| Local Hardware | M2/M3/M4 Mac | M2/M3/M4 Mac | High-end GPU |
| Pricing | Open Weights | Open Weights | Proprietary API |
๐ ๏ธ Technical Deep Dive
- Architecture: Mixture-of-Experts (MoE) with 35B total parameters, utilizing a sparse activation mechanism where only a subset of parameters are active per token inference.
- Quantization: UD-Q5_K_XL utilizes a hybrid bit-width approach, applying higher precision to attention heads and lower precision to feed-forward network layers to maintain perplexity.
- Context Handling: Implements RoPE (Rotary Positional Embeddings) with base frequency scaling to support 128k context without requiring fine-tuning for specific sequence lengths.
- API Compatibility: The llama.cpp server implementation maps the model's internal logit outputs to the OpenAI Chat Completions API schema, enabling seamless integration with tools like 'pi' or 'Continue'.
๐ฎ Future ImplicationsAI analysis grounded in cited sources
On-device MoE models will become the standard for local coding assistants.
The efficiency gains from sparse MoE architectures allow high-parameter performance on consumer-grade unified memory hardware.
Quantization techniques will increasingly target specific hardware memory controllers.
The success of UD-Q5_K_XL demonstrates that hardware-aware quantization provides significant latency improvements over generic GGUF formats.
โณ Timeline
2025-09
Alibaba releases Qwen3.0 series, introducing native long-context support.
2026-01
Qwen3.5 update introduces improved MoE routing efficiency.
2026-03
Qwen3.6 series launch, featuring optimized 35B-A3B architecture.
๐ฐ
Weekly AI Recap
Read this week's curated digest of top AI events โ
๐Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/LocalLLaMA โ