Qwen3 9B runs 6+ t/s on Android phones

🔑 Enhanced Key Takeaways

• •Qwen3.5-9B employs a hybrid architecture combining Gated DeltaNet with Sparse MoE, using a 3:1 ratio of linear to softmax attention for reduced memory and compute costs[1][3][7].
• •The model supports a 262K token context window and native multimodality, processing text and visual data in a unified latent space[2][6][7].
• •Qwen3.5-9B outperforms prior Qwen3-30B on MMLU and math benchmarks like GSM8K/MATH due to Scaled RL training[1][3].
• •At Q4 quantization, it runs on ~5GB RAM with CPU-only inference at 20-30 t/s, and higher speeds like 115-167 t/s in optimized desktop tests[3][5].

🛠️ Technical Deep Dive

• •Hybrid architecture: Gated DeltaNet + Sparse Mixture-of-Experts (MoE), with 3:1 linear attention to softmax attention ratio, reducing computational cost for long contexts[3][7].
• •Parameter count: 9.65B, natively multimodal vision-language model[6][7].
• •Training: Scaled Reinforcement Learning (RL) optimizes logical reasoning, closing gap with 30B+ models[1][2][3].
• •Context window: 262K tokens[6].
• •Quantized (Q4 GGUF): ~5GB RAM footprint, supports CUDA/NVIDIA GPU, Metal/Apple Silicon, or CPU inference[3].

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline