How GPU Cloud Pricing Works: On-Demand vs Spot vs Reserved
On-demand, spot, or reserved — three ways to buy the same GPU hour at very different prices. This plain-English guide explains when each model makes sense, how billing actually works, and what else quietly shapes the final bill.
All prices in this guide are illustrative examples for education. Real GPU cloud prices vary by provider, region, and availability and change over time. Always confirm current pricing with a provider before budgeting or committing.
On this page
What is a "GPU hour"?
Nearly every GPU cloud bills in GPU-hours: one GPU running for one hour equals one GPU-hour. If you run 4 GPUs for 6 hours, you have consumed 24 GPU-hours. The price you pay is the GPU-hour count multiplied by the hourly rate for the GPU type and pricing model you chose. This simple unit is the foundation of everything else, so it is worth internalizing: your job when budgeting is to estimate GPU-hours first, and only then worry about the rate.
Providers usually advertise a per-GPU hourly rate. A machine with 8 GPUs is typically billed at 8× the single-GPU rate (sometimes with a small discount for full nodes, sometimes not). Storage, networking, and data transfer are usually billed separately — more on that below.
On-demand pricing
On-demand is the default: you launch a GPU instance whenever you want, pay the standard hourly rate, and shut it down when you are done. No commitment, no interruption risk, no upfront payment. It is the most flexible and therefore the most expensive way to rent GPUs.
Use on-demand when your workload is interactive or time-sensitive: debugging a training script, running a demo, serving inference traffic that cannot tolerate interruptions, or doing short experiments where convenience matters more than cost. For example, an illustrative on-demand rate for a high-end GPU might be shown as $2.00/hour in teaching examples — your real rate will differ, and the point of the example is only to show the math, not the market.
The main risk of on-demand is not price volatility — the rate is stable — but availability. During periods of high demand, the GPU type you want may simply be unavailable in your preferred region. Popular GPUs can and do sell out. If your work cannot wait, you may need to try another region or a different GPU.
Spot / preemptible pricing
Spot instances (called "preemptible" on some clouds) let you rent spare GPU capacity at a steep discount — often illustrated in teaching materials as 50–70% below on-demand, though the real discount varies. The catch: the provider can reclaim the instance at any time, usually with a short warning (commonly around 30 seconds to a couple of minutes, depending on the provider).
Spot is ideal for fault-tolerant batch workloads: training runs with frequent checkpointing, hyperparameter sweeps where each trial is independent, data preprocessing, and batch inference. If an instance is reclaimed, you lose only the work since your last checkpoint, then resume on a new instance.
Spot is a poor fit for anything stateful or interactive: live inference endpoints, long-running notebooks you are actively editing, or training jobs that cannot checkpoint. A common beginner mistake is running an interactive development session on spot and losing unsaved work to a preemption.
To use spot well, build checkpointing into your workflow from day one: save model weights and optimizer state regularly to persistent storage, make your startup script idempotent (able to resume cleanly), and automate re-requesting capacity. With those habits, spot becomes the single biggest lever for cutting GPU costs.
Reserved / committed pricing
Reserved or committed-use pricing trades commitment for a lower rate. You agree to use (or pay for) a certain amount of GPU capacity for a fixed term — commonly one or three years — and in exchange the hourly rate drops, often illustrated as roughly 30–50% below on-demand in teaching examples.
This model suits steady, predictable workloads: a production inference service with stable traffic, a research lab that trains models continuously, or a company whose GPU usage never drops below a known floor. The key discipline is to reserve only your baseline usage — the amount you are confident you will consume — and handle peaks with on-demand or spot capacity.
The risk is over-commitment. If your project ends, pivots, or moves to a different GPU, you may still be paying for capacity you no longer need. Some providers allow you to exchange or resell commitments, but terms vary widely. As a rule of thumb, do not sign a multi-year commitment until you have several months of usage data showing a stable baseline.
Comparing the three models
| On-demand | Spot / preemptible | Reserved / committed | |
|---|---|---|---|
| Illustrative relative price | Baseline (highest) | Often much lower | Lower than on-demand |
| Interruption risk | None | Can be reclaimed anytime | None |
| Commitment | None | None | 1–3 years typical |
| Availability | Usually good, can sell out | Least reliable | Guaranteed capacity |
| Best for | Interactive work, inference, deadlines | Checkpointed batch jobs, sweeps | Steady production workloads |
Note: the "relative price" row describes the typical direction of the discount, not a guaranteed figure. Actual discounts vary by provider, GPU type, region, and time.
What else affects the price
The headline GPU rate is not the whole bill. When comparing providers, look at these factors too:
- Region. The same GPU can cost different amounts in different data-center regions, reflecting local demand, power costs, and capacity. Some developers route flexible workloads to cheaper regions — but weigh that against latency and data-transfer costs.
- GPU scarcity. New flagship GPUs often carry a premium and sell out; previous-generation cards are usually cheaper and more available. A slightly older GPU at a much lower rate can be the better value for many workloads.
- Storage. Fast local SSDs, persistent disks, and object storage are billed separately — usually per gigabyte per month. Training datasets and checkpoints can make storage a meaningful line item.
- Data transfer (egress). Moving data out of a cloud (egress) is commonly charged per gigabyte, while ingress is often free. If your workflow downloads large models or datasets repeatedly, egress adds up.
- Minimum billing increments. Some providers bill per second, others per minute or per hour. A provider with per-hour minimums penalizes short experiments; per-second billing favors them.
- Idle time. You pay for the instance while it exists, not just while the GPU is computing. An instance left running overnight with no job on it bills the same as one training at full speed. Shutting down promptly is free money.
How billing actually works
A typical billing cycle works like this: the provider meters your GPU usage continuously, rounds each usage segment up to its billing increment, multiplies by your rate, and adds storage and networking charges. You receive an itemized invoice, usually monthly, and most providers offer a dashboard with near-real-time spend tracking.
Two habits will save you from surprise bills. First, set up billing alerts before you launch anything — a daily or weekly spend notification takes minutes to configure and catches runaway jobs early. Second, tag or label your resources by project so you can see which experiment cost what. "The GPU bill was $400" is useless information; "the LoRA sweep cost $310 and the baseline cost $90" is actionable.
Also understand the difference between stopping and terminating an instance. Stopping usually halts billing for the GPU but may continue billing for attached storage; terminating releases everything. And some providers bill for a reserved IP address or load balancer even when no GPU is running. Read the provider's billing docs for the exact rules — they differ.
Which model should you choose?
For most developers learning the ropes, a sensible progression is: start with on-demand while you are experimenting and your usage is unpredictable; move suitable batch work to spot once you have checkpointing in place; and consider reserved capacity only after months of data show a stable baseline. There is no prize for optimizing too early — the biggest cost wins come from not leaving instances idle and from matching the GPU to the workload, topics covered in our other guides.
From examples to real prices
Everything above teaches you the structure of GPU pricing. The actual numbers live on provider pricing pages — and because we don't sell compute, we don't quote them. Instead, use our GPU cloud provider directory: official pricing pages for 18 GPU clouds, organized by type, all verified to load. Pick 3–4 providers, note the per-GPU hourly rate for your region and pricing model, then run the numbers through the cost calculator with your estimated GPU-hours (see how to estimate them). That's a budget built on real data, not on a table that went stale the week it was published.
Verifying any quote against primary sources
A GPU price is only comparable when five things match: the exact GPU model, the region, the billing unit (per GPU-hour or per instance-hour), the commitment term, and what else is bundled into the instance, such as CPU, RAM and local storage. Providers publish their own reference points: Amazon EC2 on-demand pricing, Google Cloud GPU pricing and Microsoft Azure virtual machine pricing. Marketplace prices will differ from these, sometimes by a lot, but the provider pages tell you which instance a marketplace offer corresponds to.
Frequently asked questions
Why do two listings for the same GPU differ so much? Usually because one of the five matching points above differs: a different region, a spot rather than on-demand term, or an instance with much less CPU and storage attached.
Should I compare per-GPU or per-instance prices? Convert everything to cost per GPU-hour first, then add the storage and data-transfer lines from your last invoice, so the comparison covers the whole bill rather than the headline rate.