NVIDIA / Ampere
A100 SXM 80GB
SXM4 A100 80GB variant. Server-board and thermal configuration affect system-level requirements.
- Memory
- 80 GB HBM2e
- Bandwidth
- 2,039 GB/s
- Power
- 400 W standard configuration
GPU SELECTION / AI INFRASTRUCTURE
Choose a GPU by workload, not model name alone.
Compare manufacturer-published specifications for exact product variants. This guide is a technical reference, not an inventory, price list or guarantee of availability. Confirm the server, driver, software, cooling and support configuration before procurement.
PUBLISHED GPU RECORDS
Manufacturer source and verification date
Each record links to the cited manufacturer page. Specifications can differ between PCIe, SXM, server and board variants.
| Accelerator | Architecture | Memory | Bandwidth | Power | Reviewed | Sources |
|---|---|---|---|---|---|---|
| NVIDIA A100 SXM 80GB | Ampere | 80 GB HBM2e | 2,039 GB/s | 400 W standard configuration | 2026-10-17 | Manufacturer page Record details |
| NVIDIA H100 SXM 80GB | Hopper | 80 GB HBM3 | 3.35 TB/s | Up to 700 W (configurable) | 2026-10-17 | Manufacturer page Record details |
| NVIDIA L40S 48GB | Ada Lovelace | 48 GB GDDR6 ECC | 864 GB/s | 350 W | 2026-10-17 | Manufacturer page Record details |
| AMD Instinct MI300X OAM | CDNA 3 | 192 GB HBM3 | 5.3 TB/s | 750 W typical board power | 2026-10-17 | Manufacturer page Record details |
| NVIDIA L4 24GB | Ada Lovelace | 24 GB GDDR6 | 300 GB/s | 72 W TDP | 2026-10-17 | Manufacturer page Record details |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | Blackwell | 96 GB GDDR7 ECC | 1,597 GB/s | Up to 600 W (configurable) | 2026-10-17 | Manufacturer page Record details |
NVIDIA / Hopper
H100 SXM 80GB
H100 SXM 80GB variant. Confirm the supported server platform, power envelope and cooling design.
- Memory
- 80 GB HBM3
- Bandwidth
- 3.35 TB/s
- Power
- Up to 700 W (configurable)
NVIDIA / Ada Lovelace
L40S 48GB
L40S PCIe accelerator. Confirm chassis airflow, power and workload compatibility.
- Memory
- 48 GB GDDR6 ECC
- Bandwidth
- 864 GB/s
- Power
- 350 W
AMD / CDNA 3
Instinct MI300X OAM
MI300X OAM accelerator. Validate the complete platform and ROCm software compatibility.
- Memory
- 192 GB HBM3
- Bandwidth
- 5.3 TB/s
- Power
- 750 W typical board power
NVIDIA / Ada Lovelace
L4 24GB
Low-profile single-slot accelerator for video, AI and graphics workloads. Verify supported server airflow and software versions.
- Memory
- 24 GB GDDR6
- Bandwidth
- 300 GB/s
- Power
- 72 W TDP
NVIDIA / Blackwell
RTX PRO 6000 Blackwell Server Edition
Server Edition accelerator. Power and cooling depend on the selected air-cooled or liquid-cooled server configuration.
- Memory
- 96 GB GDDR7 ECC
- Bandwidth
- 1,597 GB/s
- Power
- Up to 600 W (configurable)
Specifications are transcribed from the linked manufacturer sources and reviewed on the date shown. Check the exact SKU, system certification, software compatibility, operating requirements, thermals and power before using these values in a BOM.
WORKLOAD SELECTION
Decide what to benchmark
Video inference and analytics
Confirm codecs, resolution, frame rate, streams per server and the combined decode, model and encode pipeline. Compare throughput and watts per stream on the exact system.
LLM inference
Check model format, usable VRAM, context length, KV cache, concurrent requests, first-token latency and token throughput. Weight memory alone is not a fit guarantee.
Training and fine-tuning
Review model strategy, memory, multi-GPU interconnect, storage/checkpoint traffic, framework support and collective performance on the target node.
Scientific computing
Check the solver's supported GPU, precision requirements, MPI/runtime, memory per case, vendor validation guidance and scaling behavior.
Workstation and visual computing
Match the exact workstation/server edition to application certification, display/render requirements, cooling, slot, power and driver needs.
Sharing and isolation
Separate software-level sharing, queues and container limits from hardware-backed GPU partitioning. Verify the feature on the selected SKU.
VIDEO WORKLOADS
Codec engines and AI compute are different parts of the pipeline.
No one GPU is best for every video task. Test codec support, input mix, model size, stream count, latency and encode/decode concurrency together.
| Model | Manufacturer-listed video features | Planning checks | Official source |
|---|---|---|---|
| L4 | 24 GB memory, 72 W TDP, 2 NVENC and 4 NVDEC engines. | Low-profile single-slot form factor; validate codec, stream count, server airflow, model memory and end-to-end latency. | NVIDIA L4 specifications |
| L40S | 48 GB GDDR6 ECC, 864 GB/s, 350 W, 3 NVENC and 3 NVDEC; AV1 encode/decode support. No MIG or NVLink for this model. | Passive dual-slot card; confirm host airflow, power, video pipeline and required GPU sharing behavior. | NVIDIA L40S specifications |
| RTX PRO 6000 Blackwell Server Edition | 96 GB GDDR7 ECC, 1,597 GB/s, up to 600 W configurable; 9th-generation NVENC and 6th-generation NVDEC. | Confirm exact air or liquid-cooled system design, codec profile, thermals, power and runtime support. | NVIDIA RTX PRO 6000 Server Edition |
Engine counts and capabilities are manufacturer specifications; actual pipeline throughput depends on codecs, presets, resolution, model and server configuration.
LLM WEIGHT MEMORY ESTIMATE
A quick estimate before runtime overhead.
Approximate parameter weight memory = parameter count × bytes per parameter. The values below use decimal GB and assume one stored value per parameter.
| Parameter count | FP32 (4 bytes) | FP16 / BF16 (2 bytes) | INT8 (1 byte) | INT4 (0.5 byte) |
|---|---|---|---|---|
| 7 billion | 28 GB | 14 GB | 7 GB | 3.5 GB |
| 70 billion | 280 GB | 140 GB | 70 GB | 35 GB |
This estimate excludes KV cache, activations, framework/runtime buffers, quantization scales and metadata, fragmentation, reserved memory and other processes. Quantized formats do not always use exactly the nominal bit count. A model fitting by weight size alone does not guarantee startup, target context length, concurrency or performance.
For NVIDIA NIM profiles, the model profile guidance explains the relevant runtime memory considerations. Review its current release alongside the chosen model.
PLAN A VALIDATED CONFIGURATION
Share the model, pipeline and server constraints.
Include model/version, format, context, concurrency, codecs, software versions, power/cooling and target performance. Unknowns can be marked for evaluation.