NVGPU

Compute, considered.

GPU fundamentals: memory, throughput and cloud configurations

Understand GPU terminology and the differences that matter when reading a cloud rental listing.

By NVGPU · Substantively reviewed September 26, 2026 · Published documentation and calculations; no NVGPU hardware benchmark.

What does GPU mean?

GPU stands for graphics processing unit. GPUs perform many operations in parallel, which makes them useful for graphics and suitable parallel compute workloads. CPUs handle orchestration and general-purpose work; a GPU does not accelerate every program automatically. NVIDIA's CUDA programming guide describes its GPU programming model.

VRAM, RAM and bandwidth are different

VRAM is the GPU's local memory. Host RAM belongs to the CPU system. Neither is the same as disk storage. Capacity determines what data can fit; bandwidth describes how much data can move per unit time under specified conditions. High bandwidth does not guarantee fast application execution.

Multi-GPU totals can be misleading: four GPUs with 24 GB each do not automatically behave as one 96 GB GPU. Software must distribute work and data. A partition with reduced VRAM is also different from exclusive access to a full GPU.

What do TFLOPS and Tensor Core peaks mean?

A TFLOPS figure expresses trillions of floating-point operations per second under a stated arithmetic format. FP32, TF32, FP16, BF16 and FP8 are different formats; Tensor Core peaks and ordinary arithmetic throughput are not interchangeable. Figures with structured sparsity also have specific data requirements.

Use theoretical peaks to understand architecture, then measure your model, precision, batch size and latency target. Our reference table names the hardware variant and sparsity condition. It does not convert peaks into tokens per second.

Why PCIe, SXM and NVL matter

A family name such as H100 is not a complete configuration. Packaging, power limits, memory and interconnect can differ. NVIDIA's H100 specifications distinguish SXM and NVL. Our catalog preserves an explicit variant from source names and labels the rest unspecified. Confirm the exact SKU and topology with the provider.

Read a rental row in this order

  1. GPU model, reported variant and VRAM per GPU.
  2. GPU count and full instance cost, not just normalized per-GPU rate.
  3. Region, CPU, host RAM and reported disk.
  4. On-demand or interruptible pricing, billing minimum and commitment.
  5. Source publication date, provider quote and unpriced extras.

Per-GPU hourly price is the full-instance hourly price divided by GPU count. Check the rental checklist.