Qwen3.5-397B Uncensored NVFP4 Released

🔑 Enhanced Key Takeaways

• •Qwen3.5-397B-A17B is a native multimodal model from Alibaba's Qwen team, supporting text, image, and video inputs via early fusion training, achieving top benchmarks like 87.8% MMLU-Pro and 88.6% MathVision[1][2][3].
• •Features Hybrid MoE architecture with 397B total parameters but only 17B active per token, enabling 8.6-19x faster decoding than Qwen3-Max at long contexts[1][2][4].
• •Released openly by QwenLM on GitHub on 2026-02-16 as the first in Qwen3.5 series, ranking #3 on Artificial Analysis Intelligence Index with score of 45[3][9].
• •Available via NVIDIA NIM and Together AI APIs with FP4 quantization option, priced at $0.60 input / $3.60 output per million tokens on Together[1][2].

🛠️ Technical Deep Dive

• •Architecture: Transformer Causal LM with Vision Encoder; Hybrid MoE using Gated DeltaNet (64 linear attention heads for V, 16 for QK, head dim 128) + Gated Attention MoE; 60 layers, hidden dim 4,096, vocab 248,320[1].
• •Efficiency: Sparse MoE with 10 routed + 1 shared expert out of 512; multi-token prediction (MTP); YaRN RoPE scaling to 1M tokens; 256K native context[1][2].
• •Inputs: Unified multimodal (text, RGB images, MP4/WebM video); ViT encoder fused early with LM; supports 1D/2D/3D parameters[1].
• •Performance: 8.6x faster at 32K, 19x at 256K vs Qwen3-Max; multilingual math 73.3% PolyMATH; instruction 76.5% IFBench[2][4].

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline