Qwen3.5-9B GGUF Quant Rankings by KLD

🔑 Enhanced Key Takeaways

• •Unsloth's March 5th 2026 update enhanced quantization for Qwen3.5 MoEs, reducing Maximum KLD significantly beyond 99.9% metrics by improving outlier handling[2].
• •Qwen3.5-9B abliterated (uncensored) GGUF versions underperform even Q4_K_L quants at Q6_K levels, showing poor preservation of capabilities post-abliteration[1].
• •Imatrix calibration substantially improves low-bit quantization performance across all Unsloth quants, particularly reducing KLD for sensitive tensors like ssm_out at 2 bits[2].
• •Attn_* tensors and ssm_out are highly sensitive to heavy quantization in Qwen3.5's hybrid architecture, recommending higher precision to minimize KLD spikes[2].

🛠️ Technical Deep Dive

• •Architecture: Dense Transformer Decoder with 32 layers, hidden dimension 4096, 32 attention heads (16 for QK), gated Delta Networks + sparse MoE for hybrid efficiency[3][4].
• •Context: 128K tokens, vocabulary ~150K, supports FP16/INT8/INT4 precisions, consumer GPU compatible (RTX 3060/4060 quantized)[3].
• •Quant sensitivity: ffn_up_exps/ffn_gate_exps tolerate 3-bit; attn_* and ssm_out/*beta/alpha highly sensitive—avoid heavy quant or MXFP4 (worse than Q4_K at 4.5 bits)[2].
• •Embed/output: Some quants (Q3_K_XL, Q4_K_L) use Q8_0 for embeddings/outputs instead of defaults[6].

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline