Why SoftBank chooses NVIDIA for its AI supercomputer

Read original on ITmedia AI+ (日本)

#gpu#supercomputer#hardwarenvidia-gpusoftbanknvidiacuda

Understand the real-world engineering trade-offs behind building a 122 billion yen AI supercomputer.

30-Second TL;DR

What Changed

SoftBank invested 122 billion yen in AI infrastructure

Why It Matters

Reinforces NVIDIA's dominance in the high-performance computing market. It provides insight into the trade-offs between cost and performance for large-scale AI infrastructure.

What To Do Next

Evaluate the long-term maintenance and ecosystem support costs when selecting hardware for large-scale AI clusters.

Who should care:Developers & AI Engineers

Key Points

•SoftBank invested 122 billion yen in AI infrastructure
•NVIDIA GPUs selected for superior ecosystem and reliability
•Developers address initial hardware stability concerns

Deep Insight

AI-generated analysis for this event — not the original article.

Enhanced Key Takeaways

•SoftBank's supercomputer project is part of a broader strategic initiative to build a sovereign AI infrastructure in Japan, reducing reliance on overseas cloud providers.
•The infrastructure utilizes NVIDIA's Blackwell architecture, specifically leveraging the GB200 NVL72 platform to handle massive parameter counts for large language models.
•SoftBank is integrating its proprietary AI models, such as the 'SBCM' (SoftBank Corp Model), directly into this hardware stack to optimize inference performance.
•The investment includes a significant focus on liquid cooling technology to manage the thermal output of high-density GPU clusters, which was a primary driver for the initial hardware stability challenges.
•This initiative aligns with the Japanese government's 'AI and Software-Defined Infrastructure' policy, which provides subsidies for domestic companies building large-scale compute clusters.

Competitor Analysis

Primary Hardware

SoftBank (NVIDIA-based): NVIDIA GB200 NVL72
Competitor (e.g., AWS/Google): Custom Silicon (Trainium/TPU)
Local Japanese Cloud Providers: NVIDIA H100/H200 Clusters

Control

SoftBank (NVIDIA-based): Sovereign/Private
Competitor (e.g., AWS/Google): Public Cloud/Shared
Local Japanese Cloud Providers: Sovereign/Shared

Latency

SoftBank (NVIDIA-based): Ultra-low (Dedicated)
Competitor (e.g., AWS/Google): Variable (Network dependent)
Local Japanese Cloud Providers: Low (Regional)

Feature	SoftBank (NVIDIA-based)	Competitor (e.g., AWS/Google)	Local Japanese Cloud Providers
Primary Hardware	NVIDIA GB200 NVL72	Custom Silicon (Trainium/TPU)	NVIDIA H100/H200 Clusters
Control	Sovereign/Private	Public Cloud/Shared	Sovereign/Shared
Latency	Ultra-low (Dedicated)	Variable (Network dependent)	Low (Regional)

Technical Deep Dive

Architecture: Utilizes NVIDIA GB200 NVL72, connecting 72 Blackwell GPUs and 36 Grace CPUs in a single rack.
Interconnect: Employs 5th Gen NVLink for 1.8TB/s bidirectional bandwidth per GPU.
Cooling: Implementation of direct-to-chip liquid cooling systems to support high TDP (Thermal Design Power) per rack.
Software Stack: Integration of NVIDIA AI Enterprise software suite for orchestration, including NeMo for model training and TensorRT-LLM for inference optimization.

Future ImplicationsAI analysis grounded in cited sources

SoftBank will achieve a 30% reduction in inference costs for its internal AI services by 2027.

The transition to Blackwell-based hardware significantly improves energy efficiency and tokens-per-watt metrics compared to previous generation clusters.

SoftBank will launch a commercial 'AI-as-a-Service' platform for Japanese enterprises by Q4 2026.

The scale of the 122 billion yen investment exceeds internal requirements, necessitating a commercialization strategy to recoup capital expenditures.

Timeline

2023-11

SoftBank announces plans to build a large-scale generative AI computing platform in Japan.

2024-05

SoftBank confirms the expansion of its AI compute infrastructure, targeting the deployment of NVIDIA's latest GPU architectures.

2025-02

SoftBank begins initial testing of high-density liquid-cooled server racks to address thermal management issues.

2026-01

SoftBank officially integrates the 122 billion yen supercomputer cluster into its production environment.

2023-11
SoftBank announces plans to build a large-scale generative AI computing platform in Japan.
2024-05
SoftBank confirms the expansion of its AI compute infrastructure, targeting the deployment of NVIDIA's latest GPU architectures.
2025-02
SoftBank begins initial testing of high-density liquid-cooled server racks to address thermal management issues.
2026-01
SoftBank officially integrates the 122 billion yen supercomputer cluster into its production environment.

Read original article on ITmedia AI+ (日本)

Weekly AI Recap

Read this week's curated digest of top AI events →

Post LinkedIn