🦙Reddit r/LocalLLaMA•Feb 27, 2026Stalecollected in 8h

Qwen3.5-35B Dynamic GGUFs Hit SOTA Benchmarks

Post LinkedIn

🦙Read original on Reddit r/LocalLLaMA

#quantization#benchmarks#kl-divergenceqwen3.5-35b-a3b-unsloth-dynamic-ggufsqwen3.5-35bunslothgguf

💡SOTA quantized Qwen3.5 GGUFs + 9TB benchmarks for local LLMs

⚡ 30-Second TL;DR

What Changed

SOTA 99.9% KL Divergence on UD-Q4_K_XL, IQ3_XXS

Why It Matters

Improves efficiency for local inference of large models like Qwen3.5, enabling better quantized performance without quality loss. Community gains extensive benchmarks for future quant work.

What To Do Next

Download updated Qwen3.5-35B-A3B GGUFs and re-quantize with Imatrix for SOTA perplexity.

Who should care:Researchers & Academics

Key Points

•SOTA 99.9% KL Divergence on UD-Q4_K_XL, IQ3_XXS
•Retiring MXFP4 from most Q2_K_XL, Q3_K_XL, Q4_K_XL quants
•Imatrix reduces KLD & PPL; I-quants 5-10% slower
•9TB artifacts available; sensitive tensors like ssm_out avoided

🧠 Deep Insight

Background and context from public sources — not the original article. 6 sources cited.

🔑 Enhanced Key Takeaways

•Qwen3.5-35B-A3B features 35B total parameters with only 3B activated, using a hybrid architecture of Gated Delta Networks and sparse Mixture-of-Experts (256 experts, 8 routed + 1 shared active) for efficient inference[2].
•Model scores 37 on Artificial Analysis Intelligence Index, significantly above the median of 15 for similar open-weight models, and generates output at 166 tokens per second via Alibaba's API[1].
•Supports native 262,144 token context length, vision-language capabilities with early fusion training, and expanded coverage of 201 languages and dialects[2].

🛠️ Technical Deep Dive

•Hybrid architecture: Gated Delta Networks + sparse MoE with 256 total experts (8 routed + 1 shared active) for high-throughput inference and low latency[2].
•Reasoning model using extended chain-of-thought: Enable via 'Enable Thinking' boolean parameter; supports step-by-step reasoning display in APIs like OpenRouter[2][5].
•Multimodal: Native vision-language with early fusion training on multimodal tokens, outperforming prior Qwen3-VL on reasoning, coding, agents, and visual benchmarks[2].
•Performance: 166.1 t/s output speed (above median 90.9 t/s); generated 100M tokens on Intelligence Index eval (high vs. median 12M)[1].
•RL training: Scaled across million-agent environments with complex task distributions for real-world adaptability[2].

🔮 Future ImplicationsAI analysis grounded in cited sources

Qwen3.5-35B-A3B enables cost-effective local deployment of advanced reasoning on consumer hardware

Its 3B activated parameters from 35B total allow fitting into 24GB VRAM at high quantizations like AQ8 while outperforming larger dense models[4].

Drives adoption of dynamic quants in local inference ecosystems

SOTA KL Divergence on UD-Q4_K_XL and IQ3_XXS plus 9TB artifacts provide benchmarks and configs to standardize high-fidelity low-bit quantization[article].