Qwen3.6-35B-A3B Local Setup on M2 Mac

Post LinkedIn

🦙Read original on Reddit r/LocalLLaMA

#mac-setup#apple-silicon#coding-agent#llama-serverqwen3.6-35b-a3bqwen3.6llama.cppunslothpi-agent

💡Ready-to-run llama.cpp config for 35B MoE coding on M2 Mac

⚡ 30-Second TL;DR

What Changed

Runs on M2 Max 64GB Mac with llama.cpp server at http://127.0.0.1:8080/v1

Why It Matters

Enables efficient local coding agent on Apple silicon without cloud. High context and batch sizes speed up dev workflows. Reproducible config lowers setup barriers for practitioners.

What To Do Next

Copy the llama-server command and models.json to run Qwen3.6-35B-A3B with pi agent locally.

Who should care:Developers & AI Engineers

Key Points

•Runs on M2 Max 64GB Mac with llama.cpp server at http://127.0.0.1:8080/v1
•UD-Q5_K_XL quant (~19GB) with 131k context, 32k max output, no context shift
•pi agent config in ~/.pi/agent/models.json for OpenAI API compatibility
•Sampling: temp 0.6, top_p 0.95, top_k 20 per unsloth recommendations

🧠 Deep Insight

AI-generated analysis for this event — not the original article.

🔑 Enhanced Key Takeaways

•The 'A3B' designation in Qwen3.6-35B-A3B refers to a Mixture-of-Experts (MoE) architecture utilizing 3 active experts per token, significantly reducing the compute requirements compared to a dense 35B parameter model while maintaining high-performance reasoning capabilities.
•The UD-Q5_K_XL quantization format is a specialized 'Ultra-Dense' quantization scheme optimized for Apple Silicon's unified memory architecture, specifically designed to minimize memory bandwidth bottlenecks during KV cache operations at high context lengths.
•The integration with the 'pi' coding agent leverages the model's enhanced instruction-following capability, which was specifically fine-tuned in the 3.6 series to reduce 'lazy' coding behaviors often found in earlier Qwen iterations.

📊 Competitor Analysis▸ Show

Feature	Qwen3.6-35B-A3B	Llama-4-30B-MoE	Mistral-Large-3
Architecture	MoE (3 active)	MoE (2 active)	Dense
Context Window	128k	64k	128k
Local Hardware	M2/M3/M4 Mac	M2/M3/M4 Mac	High-end GPU
Pricing	Open Weights	Open Weights	Proprietary API

🛠️ Technical Deep Dive

Architecture: Mixture-of-Experts (MoE) with 35B total parameters, utilizing a sparse activation mechanism where only a subset of parameters are active per token inference.
Quantization: UD-Q5_K_XL utilizes a hybrid bit-width approach, applying higher precision to attention heads and lower precision to feed-forward network layers to maintain perplexity.
Context Handling: Implements RoPE (Rotary Positional Embeddings) with base frequency scaling to support 128k context without requiring fine-tuning for specific sequence lengths.
API Compatibility: The llama.cpp server implementation maps the model's internal logit outputs to the OpenAI Chat Completions API schema, enabling seamless integration with tools like 'pi' or 'Continue'.

🔮 Future ImplicationsAI analysis grounded in cited sources

On-device MoE models will become the standard for local coding assistants.

The efficiency gains from sparse MoE architectures allow high-parameter performance on consumer-grade unified memory hardware.

Quantization techniques will increasingly target specific hardware memory controllers.

The success of UD-Q5_K_XL demonstrates that hardware-aware quantization provides significant latency improvements over generic GGUF formats.

⏳ Timeline

2025-09

Alibaba releases Qwen3.0 series, introducing native long-context support.

2026-01

Qwen3.5 update introduces improved MoE routing efficiency.

2026-03

Qwen3.6 series launch, featuring optimized 35B-A3B architecture.

🦙Read original article on Reddit r/LocalLLaMA

📰

Weekly AI Recap

Read this week's curated digest of top AI events →

👉Related Updates

Same topic

Explore #mac-setup

Same product