The physical layer

Infrastructure atlas.

A source-linked field guide to the chips, systems, and platforms behind model performance.

37 public records

Accelerators

17 of 17 records

H100 Tensor Core hardwarePhysical hardware
NVIDIAGPU

H100 Tensor Core

A Hopper data-center GPU with Tensor Cores, Transformer Engine FP8, NVLink, and MIG partitioning.

Memory

80 GB HBM3

Bandwidth

3.35 TB/s

Power

Up to 700 W

H200 Tensor Core hardwarePhysical hardware
NVIDIAGPU

H200 Tensor Core

Hopper’s memory-heavy sibling, built around 141 GB of HBM3e for larger models and KV caches.

Memory

141 GB HBM3e

Bandwidth

4.8 TB/s

Power

Up to 700 W

B200 Tensor Core hardwarePhysical hardware
NVIDIAGPU

B200 Tensor Core

Blackwell’s data-center GPU for FP4/FP6/FP8 workloads, with a large HBM3e memory pool and fifth-generation NVLink.

Memory

180 GB HBM3e

Bandwidth

Up to 8 TB/s

Compute

FP4 / FP6 / FP8

GB200 NVL72 hardwarePhysical hardware
NVIDIAASIC

GB200 NVL72

A liquid-cooled rack-scale domain that treats 72 Blackwell GPUs and 36 Grace CPUs as one tightly coupled AI system.

GPU count

72 B200

Unified memory

30 TB

GPU fabric

130 TB/s

Instinct MI300X hardwarePhysical hardware
AMDGPU

Instinct MI300X

A memory-dense OAM accelerator with 192 GB HBM3 and ROCm as its primary software ecosystem.

Memory

192 GB HBM3

Bandwidth

5.3 TB/s

Power

750 W

Instinct MI325X hardwarePhysical hardware
AMDGPU

Instinct MI325X

A CDNA 3 refresh that increases the memory pool to 256 GB HBM3e for high-parameter inference and training.

Memory

256 GB HBM3e

Bandwidth

6 TB/s

Platform

2 TB HBM3e

Instinct MI355X hardwarePhysical hardware
AMDGPU

Instinct MI355X

A fourth-generation CDNA accelerator with 288 GB HBM3e, 8 TB/s bandwidth, and native microscaling formats.

Memory

288 GB HBM3e

Bandwidth

8 TB/s

Power

1,400 W

Cloud TPU v6e / Trillium hardwarePhysical hardware
GoogleTPU

Cloud TPU v6e / Trillium

Google’s sixth-generation TPU for transformer, text-to-image, and CNN training, fine-tuning, and serving.

Memory

32 GB HBM

Bandwidth

1,638 GB/s

Pod

256 chips

Trainium2 hardwarePhysical hardware
AWSASIC

Trainium2

AWS’s second-generation training accelerator, exposed through EC2 Trn2 and UltraServer systems with Neuron software.

Memory

96 GiB HBM

Bandwidth

2.9 TB/s

Instance

16 chips

Inferentia2 hardwarePhysical hardware
AWSASIC

Inferentia2

AWS’s inference-first accelerator for cost-efficient production serving through Inf2 instances and the Neuron SDK.

Instance

Up to 16 chips

Network

Up to 100 Gbps

Software

Neuron SDK

Wafer Scale Engine 3 hardwarePhysical hardware
CerebrasASIC

Wafer Scale Engine 3

A 5 nm wafer-scale processor designed to keep model computation on one very large device instead of distributing it across GPU nodes.

AI cores

900,000

On-chip SRAM

44 GB

Peak AI

125 PFLOPS

A100 Tensor Core hardwarePhysical hardware
NVIDIAGPU

A100 Tensor Core

The established Ampere workhorse for training, inference, analytics, and HPC, still common across cloud fleets.

Memory

80 GB HBM2e

Bandwidth

2.04 TB/s

Power

Up to 400 W

L40S hardwarePhysical hardware
NVIDIAGPU

L40S

A PCIe data-center GPU for generative AI inference, rendering, video, and multimodal workloads without NVLink or MIG.

Memory

48 GB GDDR6

Bandwidth

864 GB/s

Power

350 W

L4 Tensor Core hardwarePhysical hardware
NVIDIAGPU

L4 Tensor Core

A compact, low-power PCIe accelerator for video pipelines, edge inference, and cost-sensitive production serving.

Memory

24 GB GDDR6

Bandwidth

300 GB/s

Power

72 W

GH200 Grace Hopper hardwarePhysical hardware
NVIDIAASIC

GH200 Grace Hopper

A coherent Grace CPU plus Hopper GPU package that exposes CPU memory as an extension of the GPU memory system.

GPU memory

96 GB HBM3

CPU

Up to 72 Arm cores

CPU–GPU

900 GB/s

Gaudi 3 hardwarePhysical hardware
IntelASIC

Gaudi 3

An AI accelerator designed around HBM capacity, integrated RoCE networking, and an open software stack for training and inference.

Memory

128 GB HBM2e

Bandwidth

3.67 TB/s

Compute

1,678 TFLOPS

Cloud TPU v5p hardwarePhysical hardware
GoogleTPU

Cloud TPU v5p

Google’s large-scale training TPU with 3D-torus slices and enough fabric to schedule jobs across thousands of chips.

Memory

95 GiB HBM

Bandwidth

2,765 GB/s

Pod

8,960 chips

At a glance

Accelerator comparison

ProductTypeMemory / capacityBandwidth / scaleThird signal
H100 Tensor CoreGPU80 GB HBM33.35 TB/sUp to 700 W
H200 Tensor CoreGPU141 GB HBM3e4.8 TB/sUp to 700 W
B200 Tensor CoreGPU180 GB HBM3eUp to 8 TB/sFP4 / FP6 / FP8
GB200 NVL72ASIC72 B20030 TB130 TB/s
Instinct MI300XGPU192 GB HBM35.3 TB/s750 W
Instinct MI325XGPU256 GB HBM3e6 TB/s2 TB HBM3e
Instinct MI355XGPU288 GB HBM3e8 TB/s1,400 W
Cloud TPU v6e / TrilliumTPU32 GB HBM1,638 GB/s256 chips
Trainium2ASIC96 GiB HBM2.9 TB/s16 chips
Inferentia2ASICUp to 16 chipsUp to 100 GbpsNeuron SDK
Wafer Scale Engine 3ASIC900,00044 GB125 PFLOPS
A100 Tensor CoreGPU80 GB HBM2e2.04 TB/sUp to 400 W
L40SGPU48 GB GDDR6864 GB/s350 W
L4 Tensor CoreGPU24 GB GDDR6300 GB/s72 W
GH200 Grace HopperASIC96 GB HBM3Up to 72 Arm cores900 GB/s
Gaudi 3ASIC128 GB HBM2e3.67 TB/s1,678 TFLOPS
Cloud TPU v5pTPU95 GiB HBM2,765 GB/s8,960 chips