Huawei chips successfully train DeepSeek-V4-Pro model

🔑 Enhanced Key Takeaways

•The DeepSeek-V4-Pro model is a Mixture-of-Experts (MoE) architecture featuring 1.6 trillion total parameters and 49 billion activated parameters, designed to support a substantial 1 million token context window.
•Huawei's Ascend 910C chip utilizes a dual-die packaging design, integrating two Ascend 910B SoCs, and is manufactured using SMIC's 7nm (N+2) process technology.
•DeepSeek's AI models offer native support for Huawei's Ascend processors, facilitating a straightforward conversion from CUDA to CUNN, which is vital for seamless integration of Huawei's hardware into AI development workflows.
•The United States has recently intensified its export controls, issuing new guidance to prevent Chinese companies from circumventing restrictions by accessing advanced American AI chips, such as Nvidia's Blackwell and Rubin, and AMD's MI350X, through overseas subsidiaries.
•China has officially recognized AI training and inference chips, including Huawei's Ascend 910, within its 'secure and reliable' technology assessment framework, underscoring a state-backed initiative to promote domestic alternatives.

📊 Competitor Analysis▸ Show

Feature/Metric	Huawei Ascend 910C	Nvidia H100 (Hopper)	Nvidia A100 (Ampere)
FP16 Performance	800 TFLOPS	~1000 TFLOPS (implied from 80% comparison)	320 TFLOPS
Inference Performance	~60% of H100	Higher than 910C	More suitable for LLMs beyond 10B parameters
Memory Bandwidth	3.2 TB/s	Higher than 910C (e.g., HBM3/HBM3e)	Higher than 910C
Process Technology	SMIC 7nm (N+2)	TSMC 4N (custom 5nm class)	TSMC 7nm
Logic Die Area	~60% larger than H100 for 80% performance	Smaller than 910C for higher performance	Efficient design
TDP	550 W	Prioritizes raw power, higher energy use
Interconnect	HCCL, PCIe 4.0, 512 GB/s mesh	NVLink, high die-to-die bandwidth	NVLink
Software Ecosystem	MindSpore (growing)	CUDA, TensorRT (well-established)	CUDA, TensorRT (well-established)
Training Reliability	Critical weakness for long-term training	Undisputed lead	Strong
Estimated Price	~$28,000 (similar to H100)	~$28,000

🛠️ Technical Deep Dive

DeepSeek-V4-Pro Model:
- Architecture: Mixture-of-Experts (MoE) with a novel Hybrid Attention mechanism, combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA).
- Parameters: 1.6 trillion total parameters with 49 billion activated parameters per token.
- Context Window: Supports a maximum context length of 1 million tokens.
- Training Data: Pre-trained on over 32 trillion diverse and high-quality tokens, with a focus on long documents and agentic execution traces.
- Post-training: Employs a two-stage pipeline: initial independent cultivation of domain-specific experts (via SFT and RL with GRPO), followed by unified model consolidation through on-policy distillation.
- Optimizer: Utilizes the Muon optimizer for enhanced convergence speed and training stability.
- Efficiency: Achieves significant efficiency gains, requiring only 27% of single-token inference FLOPs and 10% of KV cache compared to DeepSeek-V3.2 at a 1M-token context.
- Precision: MoE expert parameters are in FP4 precision, while most other parameters use FP8.
Huawei Ascend 910C Chips:
- Architecture: Part of the DaVinci family, integrating multiple AI cores, each comprising high-throughput matrix-multiply cube units ('AIC'), wide vector SIMD engines ('AIV'), scalar units, and multi-level scratchpad memories.
- Process Technology: Manufactured using SMIC's 7nm (N+2) process.
- Transistor Count: Features approximately 53 billion transistors.
- Packaging: Employs a dual-die packaging design, effectively co-packaging two Ascend 910B SoCs, interconnected via an organic substrate.
- Peak Throughput: Delivers 800 TFLOPS (BF16/FP16), 100 TFLOPS (FP32), and 800 TOPS (INT8) for tensor operations.
- Memory Bandwidth: Provides 3.2 TB/s memory bandwidth.
- Interconnects: Features a 512 GB/s bidirectional mesh network across AI cores, PCIe 4.0 ×16 (64 GB/s) for host-NPU communication, and Distributed HCCL (Huawei Collective Communication Library) over PCIe or TCP for multi-node clusters; Huawei Collective Communication System (HCCS) is designed to rival NVIDIA's NVLink.
- Thermal Design Power (TDP): Rated at 550 W.

🔮 Future ImplicationsAI analysis grounded in cited sources

China will significantly accelerate its domestic AI chip and model development, further reducing reliance on foreign technology.

The successful training of a large model like DeepSeek-V4-Pro on Huawei chips, coupled with tightening US export controls and China's 'secure and reliable' technology assessments, will intensify domestic efforts to achieve self-sufficiency.

Huawei's Ascend ecosystem will capture a substantial share of the AI training and inference market within China.

The demonstrated capability of Ascend 910C for complex model training, alongside native software support from major Chinese AI developers like DeepSeek, positions Huawei as a leading domestic alternative to Nvidia.

The global AI landscape will experience increased fragmentation in hardware and software ecosystems.

As China develops its own full-stack AI solutions, including chips, frameworks like MindSpore, and advanced models, it will foster a distinct ecosystem, potentially leading to reduced interoperability with Western-dominated platforms such as CUDA.

⏳ Timeline

2018

Huawei announces Ascend 910 and Ascend 310 AI chips.

2019-08

Huawei officially launches the Ascend 910 processor.

2022-10

US implements new export controls on semiconductor exports to China.

2023-07

DeepSeek, an AI company, is founded by Liang Wenfeng.

2025-03

Huawei officially launches the Atlas 900 A3 SuperPoD, incorporating Ascend 910C chips.

2026-04-24

DeepSeek-V4-Pro model is released under the MIT license.

Huawei chips successfully train DeepSeek-V4-Pro model

⚡ 30-Second TL;DR

Key Points

🧠 Deep Insight

🔑 Enhanced Key Takeaways

🛠️ Technical Deep Dive

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline

📎 Sources (24)

👉Related Updates

Chinese robot maker AgiBot pursues Hong Kong IPO