Infrastructure atlas
AWS/Inference accelerator

Inferentia2

AWS’s inference-first accelerator for cost-efficient production serving through Inf2 instances and the Neuron SDK.

ASICInferentiainferenceAWS NeuronASIC
Inferentia2 hardwareInferentia family package photo · not v2-specific

01 / Instance

Up to 16 chips

Inf2 family

02 / Network

Up to 100 Gbps

Inf2 instances

03 / Software

Neuron SDK

PyTorch and TensorFlow

Precision & peak compute

PrecisionPublished peakBasis
FP16190 TFLOPSPer chip · AWS published peak

Peak FLOPS and TOPS are theoretical hardware figures. Sparse and dense modes, chip and system totals, and form factors are not interchangeable.

AWS Inferentia2 documentation

Technical specification

Inference path
Workloads
LLM, vision, speech, recommendation
Runtime
AWS Neuron SDK
Frameworks
PyTorch and TensorFlow integrations
Instance family
EC2 Inf2
Deployment notes
Scale
Up to 16 Inferentia2 chips per instance
Networking
Up to 100 Gbps
Best fit
High-volume, lower-cost inference
Partitioning
NeuronCore-based allocation