NVIDIA Restructures Business Units, Retires Gaming Division

Blackwell Architecture (e.g., GB200, B200):

• Transistors & Process: Packs 208 billion transistors, manufactured on a custom-built TSMC 4NP process.
• Multi-Die Design: Features two reticle-limited dies connected by a high-bandwidth chip-to-chip interconnect capable of approximately 10 terabytes per second (TB/s) throughput, enabling a unified single-GPU programming model.
• Transformer Engine: Second-generation Transformer Engine with custom Tensor Core technology, supporting new precision formats including FP4 for optimized inference and training of large language models (LLMs) and Mixture-of-Experts (MoE) models.
• NVLink: Fifth-generation NVLink provides approximately 1.8 TB/s per-GPU bandwidth, enabling rack-scale NVLink domains like the GB200 NVL72. NVLink-C2C offers 900 GB/s bidirectional CPU-GPU coherency with Grace CPUs.
• Cooling: Blackwell GPUs are projected to consume up to 1 kilowatt (kW) of power each, necessitating a shift to liquid cooling for high-density deployments.
• Other Features: Includes NVIDIA Confidential Computing for secure AI, a Decompression Engine, and a Reliability, Availability, and Serviceability (RAS) Engine.

Hopper Architecture (e.g., H100/H200):

• Transistors & Process: Built with over 80 billion transistors using a TSMC 4N process.
• Tensor Cores: Fourth-generation Tensor Cores with a Transformer Engine, designed to accelerate AI models by applying mixed FP8 and FP16 precisions.
• Memory: Supports HBM3 memory, offering up to 80 GB capacity and 3 TB/s bandwidth, a 50% increase over Ampere A100.
• NVLink: Fourth-generation NVLink enhances GPU-to-GPU communication, with up to 900 GB/s total bandwidth per H100.
• Multi-Instance GPU (MIG): Allows a single GPU to be partitioned into up to seven isolated instances with dedicated resources, enhanced for multi-tenant virtualized environments.
• Confidential Computing: Introduces hardware-based security to protect application code and data in use.

CUDA Ecosystem:

• Platform: Proprietary parallel computing platform and API, officially released in 2007.
• Programming: Allows developers to program GPUs for general-purpose computing using C-like languages (C, C++, Fortran, Python).
• Libraries: Includes specialized libraries like cuDNN for deep learning, cuBLAS for linear algebra, and TensorRT for inference, which are deeply optimized for NVIDIA hardware.
• SIMT: Utilizes a Single Instruction, Multiple Threads (SIMT) approach for efficient parallel execution on GPUs.