Co-Designed Chip Cuts DeepSeek V4 Inference Costs by 75%

🔑 Enhanced Key Takeaways

•DeepSeek V4 Pro is a 1.6-trillion parameter Mixture-of-Experts (MoE) model that achieves its efficiency by activating only approximately 49 billion parameters per forward pass, significantly reducing inference compute requirements.
•The co-design effort resulted in "Day 0" optimization, ensuring DeepSeek V4 was immediately performant on Huawei Ascend hardware upon its release, facilitated by Huawei's CANN software stack with fused operators.
•Huawei's Ascend 950DT accelerator is specifically designed for the memory-intensive decode phase of large model inference, featuring 144GB of Huawei's self-developed HiZQ 2.0 HBM with 4TB/s bandwidth and a 2TB/s interconnect.
•DeepSeek V4 incorporates architectural innovations such as a Hybrid Attention Architecture (combining Compressed Sparse Attention and Heavily Compressed Attention) and Manifold-Constrained Hyper-Connections (mHC) to efficiently manage its 1-million-token context window.
•DeepSeek V4 Pro is released under the permissive MIT license, making its weights open-source and enabling self-hosting and fine-tuning, which contributes to its significantly lower API pricing compared to proprietary frontier models.

📊 Competitor Analysis▸ Show

Feature/Metric	Huawei Ascend 950DT	NVIDIA H200/B200 (for comparison)	AMD MI300 Instinct (for comparison)
Target Scenario	Training & Decode-stage Inference	Training & Inference	Training & HPC
Memory (HBM)	144GB HiZQ 2.0 HBM	141GB HBM3e (H200)	128GB HBM3 (MI300)
Memory Bandwidth	4 TB/s	4.89 TB/s (H200)	6.55 TB/s (MI300)
Interconnect Bandwidth	2 TB/s (chip-to-chip)	NVLink (system-level, higher)	Infinity Fabric (system-level, higher)
FP8 Performance	1 PFLOPS (per chip)	H200: 3.958 PFLOPS; B200: 20 PFLOPS sparse	null
FP4 Performance	2 PFLOPS (per chip)	B200: 40 PFLOPS sparse	null
Manufacturing Node	Likely SMIC N+3 (5nm-class)	TSMC 4N (H200), TSMC 4NP (B200)	TSMC 5nm (MI300)
Software Ecosystem	Huawei CANN, MindSpore	NVIDIA CUDA, cuDNN, TensorRT	AMD ROCm
Availability	Q4 2026 (950DT expected)	H200: late 2024; MI300: early 2023	MI300: early 2023

🛠️ Technical Deep Dive

DeepSeek V4 Model Architecture:
- Type: Mixture-of-Experts (MoE) architecture.
- Parameters: DeepSeek V4 Pro has 1.6 trillion total parameters, with approximately 49 billion active parameters per forward pass. DeepSeek V4 Flash has 284 billion total parameters with about 13 billion active.
- Context Window: Supports a 1-million-token context window.
- Attention Mechanism: Features a Hybrid Attention Architecture combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA).
- Inter-layer Connections: Utilizes Manifold-Constrained Hyper-Connections (mHC) to stabilize signal propagation and enhance long-context handling.
- Optimizer: Employs the Muon Optimizer for faster convergence and improved training stability.
- Precision: Uses Mixed Precision Training, with MoE expert parameters in FP4 and most other parameters in FP8, incorporating FP4 Quantization-Aware Training (QAT).
- Training: Incorporates Multi-Token Prediction (MTP) during training to improve sample efficiency and output coherence.
Huawei Ascend 950DT AI Accelerator:
- Architecture: Based on Huawei's proprietary Da Vinci architecture.
- Die Design: Features a dual-die UMA (Unified Memory Access) architecture, presented as a single device to the operating system.
- Memory: Equipped with 144GB of Huawei's self-developed HiZQ 2.0 HBM.
- Memory Bandwidth: Delivers 4 TB/s memory access bandwidth.
- Interconnect: Provides 2 TB/s interconnect bandwidth between chips.
- Data Formats: Supports multiple low-precision data formats including FP8, MXFP8, XMFP4, and HiF8.
- Software Stack: Leverages the CANN (Compute Architecture for Neural Networks) software stack for heterogeneous compute and optimization, similar to NVIDIA's CUDA.
- Optimization: Employs a three-layer parallel optimization strategy for extreme inference performance.
- Manufacturing Process: Likely manufactured on SMIC's newest N+3 node, which is a 5nm-class process.

🔮 Future ImplicationsAI analysis grounded in cited sources

The success of DeepSeek V4 on Huawei Ascend 950DT will accelerate the adoption of domestic AI hardware in China.

The 'Day 0' optimization and significant cost reduction demonstrate that Chinese firms can achieve frontier AI performance using homegrown hardware, reducing reliance on foreign technology amidst sanctions.

Hardware-software co-design will become a more critical factor for achieving optimal AI inference efficiency across the industry.

The 75% cost reduction highlights the substantial gains possible when AI models are specifically optimized for the underlying accelerator architecture, pushing the industry towards tighter integration.

DeepSeek's open-source, cost-efficient models will intensify competition in the global LLM market.

By offering frontier-level performance at significantly lower API costs and with open weights, DeepSeek challenges the pricing and proprietary nature of established Western AI models.

⏳ Timeline

2023-07

DeepSeek founded by Liang Wenfeng, funded by High-Flyer.

2023-11

DeepSeek released its first model, DeepSeek Coder.

2024-05

DeepSeek-V2 released, introducing a Mixture-of-Experts (MoE) architecture.

2024-12

DeepSeek-V3 released, a 685 billion parameter model with 37 billion active parameters.

2025-01

DeepSeek-R1 model and eponymous chatbot launched, gaining international prominence for its efficiency and open-source nature.

2026-04

DeepSeek V4 Pro (1.6T parameters) and V4 Flash (284B parameters) models released, featuring a 1M-token context window and immediate "Day 0" adaptation on Huawei Ascend 950DT.

Co-Designed Chip Cuts DeepSeek V4 Inference Costs by 75%

⚡ 30-Second TL;DR

Key Points

🧠 Deep Insight

🔑 Enhanced Key Takeaways

🛠️ Technical Deep Dive

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline

📎 Sources (27)

👉Related Updates