Chart of Top Open Models Last 6 Months

🔑 Enhanced Key Takeaways

• •The rapid proliferation of high-performance sub-30B models is being driven by advancements in speculative decoding and advanced quantization techniques (e.g., EXL2, GGUF) that allow these models to fit into consumer-grade hardware.
• •The 30-35B MoE (Mixture of Experts) models mentioned are leveraging sparse activation, where only a fraction of total parameters are active per token, enabling inference speeds that were previously impossible on 8GB VRAM.
• •The current trend in the open-weights community has shifted from pure parameter count to 'compute-optimal' training, where models like Qwen3.6 27B are trained on significantly larger datasets than their predecessors to maximize performance per parameter.

🛠️ Technical Deep Dive

• •Qwen3.6 27B utilizes a Grouped Query Attention (GQA) mechanism to reduce KV cache memory footprint, facilitating longer context windows on limited VRAM.
• •The 30-35B MoE models referenced utilize a Top-K routing mechanism where K=2, ensuring that while the total parameter count is high, the active parameter count remains low enough for real-time token generation.
• •Implementation of 4-bit and 3-bit quantization (via AutoGPTQ/AutoAWQ) has become the standard for local deployment, allowing 30B+ models to run within the 8GB-12GB VRAM envelope common in consumer GPUs.

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline