NVIDIA H100 — The Backbone of AI Compute

The NVIDIA H100 is a data-centre GPU that NVIDIA announced at GTC on 22 March 2022. It was the first product built on the new Hopper architecture, named for the US computer scientist Grace Hopper. At the announcement NVIDIA said it would be available starting in the third quarter; on 20 September 2022 it announced that the H100 was in full production, with partners planning to roll out the first wave of products in October, and with H100 instances on AWS, Google Cloud, Microsoft Azure and Oracle Cloud starting the following year. Anyone asking "when" should hold on to that gap of roughly six months between announcement and shipment.

The H100 is not a graphics card in the consumer sense. It was designed as the compute part for training and serving large language models. When ChatGPT appeared that November and demand for generative-AI computing surged, the H100 became one of the parts that absorbed it.

H100 specifications

These are the launch-time figures from NVIDIA's announcement and its architecture blog.

The full GH100 die has 144 SMs and 60 MB of L2 cache; the shipping H100 disables part of it and carries 50 MB of L2. NVIDIA's current product page lists the H100 SXM at 3.35 TB/s of memory bandwidth and 3,958 TFLOPS of FP8 Tensor Core throughput with sparsity. The H100 NVL added later, with 94 GB at 3.9 TB/s, is a different product from the PCIe card of the launch.

The Transformer Engine and FP8

The headline feature was the Transformer Engine. NVIDIA describes it as a combination of software and Hopper Tensor Core hardware designed to accelerate Transformer training and inference, choosing dynamically, layer by layer, between 8-bit floating point (FP8) and 16-bit arithmetic.

FP8 comes in two formats: E4M3, with four exponent bits and three mantissa bits, and E5M2, with five and two. Halving the bit width lets the same silicon do more operations, at a cost in precision; the Transformer Engine's job is to watch how much precision each layer can afford to lose.

Hopper also added thread block clusters, which let threads cooperate across several SMs; a Tensor Memory Accelerator (TMA) for moving large tensors; and DPX instructions for dynamic-programming algorithms.

How to read "up to 9x" and "up to 30x"

The numbers that dominated the launch were NVIDIA's own comparisons against the previous generation, the Ampere-based A100: training a 395-billion-parameter Mixture of Experts model "up to 9x faster", and chatbot inference on Megatron 530B with "up to 30x higher throughput than the previous generation". The release does not state the precision or system configuration behind either figure.

NVIDIA's architecture blog gives comparisons with clearer terms: three times the A100's FP64 and FP32 processing rate per clock, and FP8 Tensor Core throughput 6.4 times the A100's FP16. The second of those compares two different precisions. The same pattern recurred with the successor, Blackwell B200.

From announcement to shipment

At the production announcement NVIDIA expected more than 50 server models from major makers by the end of the year, opened orders for the DGX H100 with eight H100s, and bundled a five-year NVIDIA AI Enterprise licence with H100 for mainstream servers. Early adopters it named included Los Alamos National Lab, the Swiss National Supercomputing Centre (CSCS) and the University of Tsukuba.

Export controls

The H100 was a geopolitical object before it shipped. According to NVIDIA's August 2022 filing with the SEC, the US government told the company on 26 August that a new licence requirement, effective immediately, applied to any future export of the A100 and the "forthcoming H100" to China (including Hong Kong) and Russia. The stated reason was the risk of diversion to military end use, and NVIDIA said about $400 million of expected third-quarter sales to China might be affected. A follow-up filing dated 1 September said exports needed to continue H100 development had been authorised.

NVIDIA offered cut-down parts such as the H800 for China, but the October 2023 rule change brought those under licence too. The government moved the effective date from 30 days out to immediate on 23 October, and NVIDIA disclosed that, given worldwide demand, it did not expect a near-term meaningful impact on its results. In January 2025 DeepSeek-R1 was built on a base model trained on those H800s. In December 2025 President Trump was reported to have approved sales of the successor H200 to vetted Chinese customers with a 25% fee, excluding Blackwell and Rubin.

Demand, and what followed

NVIDIA's revenue for fiscal 2024 (the year to January 2024) was $60.9 billion, up 126%, with Data Center revenue up 217% to $47.5 billion. On 18 June 2024 NVIDIA briefly became the world's most valuable company (details).

The product line continued with the H200 (announced 2023) and Blackwell B200 (2024), and on 31 May 2026 NVIDIA announced that its successor platform, Vera Rubin, was ramping into full production. Underneath all of them is the parallel programming model NVIDIA first released as CUDA in 2007.

The period around it is on the hardware timeline and the AI timeline; the company and its chief executive have their own pages, NVIDIA and Jensen Huang.