Alibaba Opens Qwen3.8 Model Weights
💡A 2.4T open-weight MoE model with 256K native context is now available for experimentation.
⚡ 30-Second TL;DR
What Changed
The model contains 2.4T total parameters and activates 95B parameters per token.
Why It Matters
The release gives developers access to a frontier-scale open-weight model that may enable more customization, self-hosting, and domain adaptation. Its extreme context capacity and agent-focused training could benefit complex document and long-horizon workflow applications, though deployment costs and hardware requirements will be substantial.
What To Do Next
Download Qwen3.8-2.4T-A95B from ModelScope and benchmark its long-context inference, memory use, and agent workflows on your target hardware.
Key Points
- •The model contains 2.4T total parameters and activates 95B parameters per token.
- •It supports a native 262,144-token context window, expandable to 1,010,000 tokens.
- •Each MoE layer has 512 experts; every token selects 10 routed experts plus one shared expert.
- •Multi-Token Prediction training enables supported inference engines to predict several future tokens.
- •Weights are available through the ModelScope Qwen3.8-2.4T-A95B repository.
🧠 Deep Insight
AI-generated analysis for this event.
🔑 Enhanced Key Takeaways
- •The model employs a hybrid attention architecture where 69 of its 92 layers utilize linear attention to manage KV cache memory, while the remaining 23 layers use full attention.
- •The open-weight release is strictly text-only and mandates a 'thinking' mode, meaning all responses are automatically wrapped in
tags, unlike the hosted Qwen3.8-Max version which supports vision input and non-thinking modes. - •Alibaba's release includes a built-in Multi-Token Prediction (MTP) draft head, which allows inference engines to speculate and verify multiple future tokens simultaneously to improve generation speed.
- •The model's 2.4 trillion parameters require significant infrastructure; full-precision (BF16) weights occupy approximately 4.9TB of storage, though highly compressed 1-bit quantization variants (e.g., Unsloth's TQ1_0) can reduce this to under 400GB.
- •The release is positioned as a direct competitor to other massive MoE models like Moonshot AI's Kimi K3, with Alibaba reporting performance metrics that place it near the top of current benchmarks, including OSWorld-Verified.
📊 Competitor Analysis▸ Show
| Model | Total Parameters | Active Parameters | Context Window | Key Differentiator |
|---|---|---|---|---|
| Qwen3.8-2.4T-A95B | 2.4T | 95B | 1M | Hybrid linear/full attention; MTP-optimized |
| Moonshot Kimi K3 | 2.8T | 104B | 1M+ | High-capacity rival in the Chinese market |
| DeepSeek V4-Pro | 1.6T | ~40B | N/A | Pioneer in efficient sparse MoE scaling |
🛠️ Technical Deep Dive
- Architecture: Sparse Mixture-of-Experts (MoE) with 92 layers.
- Expert Configuration: 512 total experts; 10 routed experts and 1 shared expert active per token.
- Attention Mechanism: Hybrid design alternating between full attention and linear attention (69 linear layers, 23 full attention layers).
- Context Handling: 262,144 tokens native, extensible to 1,010,000 tokens.
- Inference Optimization: Native support for Multi-Token Prediction (MTP) and compatibility with vLLM, SGLang, and TokenSpeed.
- Precision: Supports FP8 and BF16 for standard deployment; supports extreme quantization (1-bit/1.5-bit) via custom codebooks.
🔮 Future ImplicationsAI analysis grounded in cited sources
⏳ Timeline
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: IT之家 ↗