Why China’s AI Still Depends on Nvidia

Read original on SCMP Technology

#china-ai#gpu-migration#llm-training#semiconductorsnvidia-ai-gpusnvidiacuda

China’s LLM leaders still rely on Nvidia because replacing the full software stack is prohibitively difficult.

30-Second TL;DR

What Changed

Nvidia chips remain the standard hardware for training LLMs among major Chinese AI developers.

Why It Matters

The continued reliance on Nvidia gives CUDA-based tooling and established GPU workflows lasting importance in China’s AI ecosystem. Domestic chipmakers may advance technologically but still face a significant software compatibility and migration-cost gap.

What To Do Next

Use PyTorch Profiler to benchmark your CUDA training pipeline and identify the kernels and dependencies that would need porting to a domestic accelerator.

Who should care:Researchers & Academics

Key Points

•Nvidia chips remain the standard hardware for training LLMs among major Chinese AI developers.
•Switching to domestic semiconductors requires substantial software and hardware re-engineering.
•The financial cost of replacing existing Nvidia-based infrastructure is delaying China’s AI hardware transition.

Deep Insight

AI-generated analysis for this event — not the original article.

Enhanced Key Takeaways

•The CUDA software ecosystem remains the primary 'moat' for Nvidia, as Chinese developers face immense technical debt when attempting to port proprietary CUDA-based code to heterogeneous computing frameworks like Huawei's CANN.
•U.S. export controls have forced Chinese firms to rely on 'de-tuned' chips like the H20, which offer significantly lower interconnect bandwidth compared to the H100/H200, creating a bottleneck in multi-node cluster training efficiency.
•Major Chinese cloud providers are increasingly adopting a 'hybrid' strategy, utilizing domestic NPUs for inference tasks while reserving scarce, high-performance Nvidia GPUs exclusively for the pre-training phase of frontier models.
•The lack of a unified, open-source software stack for domestic Chinese chips has led to fragmented development environments, where each manufacturer (e.g., Huawei, Cambricon, Biren) requires custom optimization for the same LLM architecture.
•Recent advancements in interconnect technologies, such as Huawei's Ascend 910C, are reportedly narrowing the performance gap, yet supply chain constraints and low manufacturing yields continue to limit large-scale deployment.

Competitor Analysis

Software Ecosystem

Nvidia (H20/H100): CUDA (Mature/Universal)
Huawei (Ascend 910C): CANN (Developing)
Cambricon (MLU590): Proprietary (Limited)

Interconnect

Nvidia (H20/H100): NVLink (Industry Std)
Huawei (Ascend 910C): Ascend Fabric (Proprietary)
Cambricon (MLU590): Proprietary

Training Throughput

Nvidia (H20/H100): High (Baseline)
Huawei (Ascend 910C): 80-90% of H100 (Claimed)
Cambricon (MLU590): Moderate

Availability

Nvidia (H20/H100): Restricted (Export Control)
Huawei (Ascend 910C): High (Domestic)
Cambricon (MLU590): High (Domestic)

Feature	Nvidia (H20/H100)	Huawei (Ascend 910C)	Cambricon (MLU590)
Software Ecosystem	CUDA (Mature/Universal)	CANN (Developing)	Proprietary (Limited)
Interconnect	NVLink (Industry Std)	Ascend Fabric (Proprietary)	Proprietary
Training Throughput	High (Baseline)	80-90% of H100 (Claimed)	Moderate
Availability	Restricted (Export Control)	High (Domestic)	High (Domestic)

Technical Deep Dive

Nvidia's H20 GPU utilizes a reduced memory bandwidth of 4.0 TB/s compared to the H100's 3.35 TB/s, but significantly lower compute density to comply with U.S. export regulations.
Huawei's CANN (Compute Architecture for Neural Networks) acts as the middleware layer attempting to bridge the gap between PyTorch/TensorFlow and Ascend hardware, though it lacks the extensive library support of CUDA.
Multi-node training in China is currently hampered by the lack of high-speed, low-latency interconnects equivalent to Nvidia's NVLink Switch System, forcing reliance on slower InfiniBand or Ethernet-based clusters.
Memory wall issues are exacerbated in domestic chips due to the limited availability of HBM3/HBM3e, forcing reliance on slower GDDR6 or HBM2e memory configurations.

Future ImplicationsAI analysis grounded in cited sources

Chinese AI firms will shift toward 'Model Distillation' as a primary strategy.

The scarcity of high-end training hardware will force developers to train smaller, more efficient models on domestic chips rather than scaling massive parameters.

The 'CUDA-lock' will persist for at least another 24 months.

The sheer volume of legacy code and the lack of a standardized, cross-vendor software abstraction layer in China prevents rapid migration.

Timeline

2022-10

U.S. Department of Commerce implements initial sweeping export controls on advanced AI chips to China.

2023-10

U.S. updates export rules, effectively banning the sale of high-performance chips like the H800 and A800 to China.

2024-02

Nvidia begins shipping the H20, a compliant, lower-performance GPU specifically designed for the Chinese market.

2025-05

Major Chinese tech giants report significant delays in LLM training timelines due to hardware supply constraints.

2026-03

Huawei announces mass production of the Ascend 910C, aiming to compete directly with Nvidia's H20 in the domestic market.

2022-10
U.S. Department of Commerce implements initial sweeping export controls on advanced AI chips to China.
2023-10
U.S. updates export rules, effectively banning the sale of high-performance chips like the H800 and A800 to China.
2024-02
Nvidia begins shipping the H20, a compliant, lower-performance GPU specifically designed for the Chinese market.
2025-05
Major Chinese tech giants report significant delays in LLM training timelines due to hardware supply constraints.
2026-03
Huawei announces mass production of the Ascend 910C, aiming to compete directly with Nvidia's H20 in the domestic market.