Run Qwen3.8-Max on GB300 with Configurable Reasoning
💡Explore how to serve a 2.4T open-weight model with configurable reasoning on GB300 NVL72.
⚡ 30-Second TL;DR
What Changed
Qwen3.8-2.4T-A95B contains 2.4 trillion total parameters, with 95 billion activated per token.
Why It Matters
This release could give developers access to a very large open-weight model while allowing them to balance reasoning depth and serving cost. Its scale also makes high-end GPU infrastructure, memory capacity, and optimized inference critical considerations for production deployment.
What To Do Next
Download the Qwen3.8-2.4T-A95B open weights and benchmark configurable reasoning on an NVIDIA GB300 NVL72 test environment before planning production capacity.
Key Points
- •Qwen3.8-2.4T-A95B contains 2.4 trillion total parameters, with 95 billion activated per token.
- •The model uses a fine-grained mixture-of-experts architecture combined with full and linear attention.
- •NVIDIA presents a serving approach on GB300 NVL72 with configurable reasoning for different inference requirements.
- •The open-weight release brings Alibaba’s near-frontier model capabilities to the broader open ecosystem.
🧠 Deep Insight
AI-generated analysis for this event.
🔑 Enhanced Key Takeaways
- •The GB300 NVL72 system utilizes high-bandwidth memory (HBM3e) to manage the massive 2.4T parameter footprint, enabling memory-efficient model parallelism across the 72-GPU rack scale.
- •Configurable reasoning in Qwen3.8-Max is achieved through a dynamic compute-allocation mechanism that allows users to adjust the number of active experts per token during inference to balance latency and accuracy.
- •Alibaba's release includes a specialized quantization kernel optimized for the Blackwell architecture, allowing the 2.4T model to run with reduced precision without significant degradation in reasoning benchmarks.
- •The model architecture incorporates a novel 'Context-Aware Routing' mechanism that improves expert selection efficiency, specifically designed to handle long-context tasks exceeding 128k tokens.
- •Deployment on GB300 NVL72 leverages NVIDIA's TensorRT-LLM library, which has been updated to support the specific MoE (Mixture-of-Experts) structure of the Qwen3.8 series for improved throughput.
📊 Competitor Analysis▸ Show
| Feature | Qwen3.8-Max (2.4T) | Llama 4-405B | DeepSeek-V3 |
|---|---|---|---|
| Architecture | MoE (95B Active) | Dense | MoE |
| Primary Use | Reasoning/Coding | General Purpose | Efficiency/Coding |
| Hardware Req | GB300 NVL72 | H100/B200 | H800/A100 |
| Open Weights | Yes | Yes | Yes |
🛠️ Technical Deep Dive
- Architecture: Fine-grained Mixture-of-Experts (MoE) with 2.4 trillion total parameters and 95 billion active parameters per token.
- Attention Mechanism: Hybrid approach utilizing full attention for short-range dependencies and linear attention for long-context windows.
- Hardware Optimization: Utilizes NVIDIA Blackwell GB300 NVL72 interconnects to minimize communication overhead during expert synchronization.
- Inference Strategy: Supports dynamic expert pruning and configurable reasoning depth, allowing users to trade off inference cost against model output quality.
- Quantization: Native support for FP8 and INT4 precision kernels optimized for Blackwell tensor cores.
🔮 Future ImplicationsAI analysis grounded in cited sources
⏳ Timeline
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: NVIDIA Developer Blog ↗