How GPU Cloud Pricing Works: On-Demand vs Spot vs Reserved
All prices in this guide are illustrative examples for education. Real GPU cloud prices vary by provider, region, and availability and change over time. Always confirm current pricing with a provider before budgeting or committing.
On this page
What is a "GPU hour"?
Nearly every GPU cloud bills in GPU-hours: one GPU running for one hour equals one GPU-hour. If you run 4 GPUs for 6 hours, you have consumed 24 GPU-hours. The price you pay is the GPU-hour count multiplied by the hourly rate for the GPU type and pricing model you chose. This simple unit is the foundation of everything else, so it is worth internalizing: your job when budgeting is to estimate GPU-hours first, and only then worry about the rate.
Providers usually advertise a per-GPU hourly rate. A machine with 8 GPUs is typically billed at 8× the single-GPU rate (sometimes with a small discount for full nodes, sometimes not). Storage, networking, and data transfer are usually billed separately — more on that below.
On-demand pricing
On-demand is the default: you launch a GPU instance whenever you want, pay the standard hourly rate, and shut it down when you are done. No commitment, no interruption risk, no upfront payment. It is the most flexible and therefore the most expensive way to rent GPUs.
Use on-demand when your workload is interactive or time-sensitive: debugging a training script, running a demo, serving inference traffic that cannot tolerate interruptions, or doing short experiments where convenience matters more than cost. For example, an illustrative on-demand rate for a high-end GPU might be shown as $2.00/hour in teaching examples — your real rate will differ, and the point of the example is only to show the math, not the market.
The main risk of on-demand is not price volatility — the rate is stable — but availability. During periods of high demand, the GPU type you want may simply be unavailable in your preferred region. Popular GPUs can and do sell out. If your work cannot wait, you may need to try another region or a different GPU.
Spot / preemptible pricing
Spot instances (called "preemptible" on some clouds) let you rent spare GPU capacity at a steep discount — often illustrated in teaching materials as 50–70% below on-demand, though the real discount varies. The catch: the provider can reclaim the instance at any time, usually with a short warning (commonly around 30 seconds to a couple of minutes, depending on the provider).
Spot is ideal for fault-tolerant batch workloads: training runs with frequent checkpointing, hyperparameter sweeps where each trial is independent, data preprocessing, and batch inference. If an instance is reclaimed, you lose only the work since your last checkpoint, then resume on a new instance.
Spot is a poor fit for anything stateful or interactive: live inference endpoints, long-running notebooks you are actively editing, or training jobs that cannot checkpoint. A common beginner mistake is running an interactive development session on spot and losing unsaved work to a preemption.
To use spot well, build checkpointing into your workflow from day one: save model weights and optimizer state regularly to persistent storage, make your startup script idempotent (able to resume cleanly), and automate re-requesting capacity. With those habits, spot becomes the single biggest lever for cutting GPU costs.
Reserved / committed pricing
Reserved or committed-use pricing trades commitment for a lower rate. You agree to use (or pay for) a certain amount of GPU capacity for a fixed term — commonly one or three years — and in exchange the hourly rate drops, often illustrated as roughly 30–50% below on-demand in teaching examples.
This model suits steady, predictable workloads: a production inference service with stable traffic, a research lab that trains models continuously, or a company whose GPU usage never drops below a known floor. The key discipline is to reserve only your baseline usage — the amount you are confident you will consume — and handle peaks with on-demand or spot capacity.
The risk is over-commitment. If your project ends, pivots, or moves to a different GPU, you may still be paying for capacity you no longer need. Some providers allow you to exchange or resell commitments, but terms vary widely. As a rule of thumb, do not sign a multi-year commitment until you have several months of usage data showing a stable baseline.
Comparing the three models
| On-demand | Spot / preemptible | Reserved / committed | |
|---|---|---|---|
| Illustrative relative price | Baseline (highest) | Often much lower | Lower than on-demand |
| Interruption risk | None | Can be reclaimed anytime | None |
| Commitment | None | None | 1–3 years typical |
| Availability | Usually good, can sell out | Least reliable | Guaranteed capacity |
| Best for | Interactive work, inference, deadlines | Checkpointed batch jobs, sweeps | Steady production workloads |
Note: the "relative price" row describes the typical direction of the discount, not a guaranteed figure. Actual discounts vary by provider, GPU type, region, and time.
What else affects the price
The headline GPU rate is not the whole bill. When comparing providers, look at these factors too:
- Region. The same GPU can cost different amounts in different data-center regions, reflecting local demand, power costs, and capacity. Some developers route flexible workloads to cheaper regions — but weigh that against latency and data-transfer costs.
- GPU scarcity. New flagship GPUs often carry a premium and sell out; previous-generation cards are usually cheaper and more available. A slightly older GPU at a much lower rate can be the better value for many workloads.
- Storage. Fast local SSDs, persistent disks, and object storage are billed separately — usually per gigabyte per month. Training datasets and checkpoints can make storage a meaningful line item.
- Data transfer (egress). Moving data out of a cloud (egress) is commonly charged per gigabyte, while ingress is often free. If your workflow downloads large models or datasets repeatedly, egress adds up.
- Minimum billing increments. Some providers bill per second, others per minute or per hour. A provider with per-hour minimums penalizes short experiments; per-second billing favors them.
- Idle time. You pay for the instance while it exists, not just while the GPU is computing. An instance left running overnight with no job on it bills the same as one training at full speed. Shutting down promptly is free money.
How billing actually works
A typical billing cycle works like this: the provider meters your GPU usage continuously, rounds each usage segment up to its billing increment, multiplies by your rate, and adds storage and networking charges. You receive an itemized invoice, usually monthly, and most providers offer a dashboard with near-real-time spend tracking.
Two habits will save you from surprise bills. First, set up billing alerts before you launch anything — a daily or weekly spend notification takes minutes to configure and catches runaway jobs early. Second, tag or label your resources by project so you can see which experiment cost what. "The GPU bill was $400" is useless information; "the LoRA sweep cost $310 and the baseline cost $90" is actionable.
Also understand the difference between stopping and terminating an instance. Stopping usually halts billing for the GPU but may continue billing for attached storage; terminating releases everything. And some providers bill for a reserved IP address or load balancer even when no GPU is running. Read the provider's billing docs for the exact rules — they differ.
Which model should you choose?
For most developers learning the ropes, a sensible progression is: start with on-demand while you are experimenting and your usage is unpredictable; move suitable batch work to spot once you have checkpointing in place; and consider reserved capacity only after months of data show a stable baseline. There is no prize for optimizing too early — the biggest cost wins come from not leaving instances idle and from matching the GPU to the workload, topics covered in our other guides.
To put numbers on your plan, try the GPU cost calculator with illustrative rates, then read how to estimate GPU hours so your inputs reflect reality. And remember: every figure on this site is a teaching example. The only prices that matter are the ones on the provider's current pricing page.