Reading a GPU Cloud Invoice: Every Line Beyond GPU-Hours
First-time GPU cloud users budget for GPU-hours, then meet the rest of the invoice. This guide walks through the line items that commonly appear, explains what each one is charging for, and flags the charges that are usually avoidable.
All figures in this guide are illustrative examples for education. Real GPU prices, cloud rates and throughput vary by provider, region, model and time. Confirm current numbers with a provider before budgeting. The point here is the method; plug in your own numbers.
On this page
The shape of a typical bill
Open a GPU cloud invoice for a month of moderate use and the GPU line is rarely alone. Illustrative breakdown for a small team doing fine-tuning and some serving:
| Line item | Illustrative share | What drives it |
|---|---|---|
| GPU instance hours | 55% | Instances running, whether busy or idle |
| Block storage (attached volumes) | 12% | Volume size × hours it exists, including detached volumes |
| Object storage | 8% | Datasets, checkpoints, model files; sometimes per-request charges |
| Data egress | 10% | Bytes leaving the provider’s network |
| Snapshots and images | 4% | Saved disk images that nobody deleted |
| Networking (IPs, load balancers) | 4% | Reserved public IPs, load balancer hours |
| CPU instances and support | 7% | Helper machines, bastions, support tiers |
Proportions vary a lot between providers; specialist GPU clouds tend to bill more simply than general clouds, which have more line items. The point is that the non-GPU lines are not rounding errors.
Instance hours: the obvious line, with a trap
You are billed for the time an instance exists in a running state, usually to the second or minute, sometimes rounded to the hour. The trap is stopped-but-not-deleted instances: on many clouds a stopped instance stops billing for the GPU but keeps billing for its attached disk and any reserved IP address. “I turned it off” does not always mean “I stopped paying”. Check the provider’s definition of stopped versus terminated.
Our guide on utilisation covers the larger problem of paying for running instances that are not doing anything.
Storage: volumes, objects and the orphan problem
Block storage (the disk attached to an instance) bills by provisioned size, not by data stored, for every hour the volume exists. A 1 TB volume that is 10 percent full costs the same as a full one. When an instance is deleted, its volume is often kept by default. Orphaned volumes from last month’s experiments are among the most common surprise charges.
Object storage bills by data stored, and frequently also by number of requests and by retrieval. Checkpoints are the usual culprit: a training job that saves every hour for three days leaves dozens of multi-gigabyte files, and nobody deletes the intermediate ones. Set a retention rule that keeps the last few and the best one.
Snapshots and machine images are copies of disks taken to preserve an environment. They are useful and they are forgotten. Audit them quarterly.
Egress: the line that scales with success
Data entering most clouds is free; data leaving is charged per gigabyte, with the rate depending on destination. Downloading a fine-tuned model to your laptop, serving large responses or files to users, replicating data to another provider, and even moving data between regions of the same provider can all count as egress.
Egress is the line that grows as your product grows, and it is the main lock-in mechanism of cloud pricing: the data you have accumulated is expensive to take elsewhere. Keep large artefacts in the region where they are used, compress what you transfer, and read the inter-region and internet egress rates before choosing where to put your data.
Networking, support and the small lines
- Reserved public IP addresses often bill hourly whether or not they are attached to anything.
- Load balancers bill by the hour plus by traffic processed. One per environment adds up.
- NAT gateways and VPN endpoints on general clouds bill hourly and per gigabyte.
- Support plans can be a percentage of total spend; check whether the tier you have is one you use.
- Logging and monitoring services bill by data ingested; verbose GPU metrics at high resolution can become a real line.
A monthly audit that takes twenty minutes
- Sort the invoice by amount. Anything you cannot explain in one sentence gets investigated.
- List every storage volume and snapshot. Delete the ones with no owner.
- Check object storage for checkpoint directories and apply a retention rule.
- List reserved IPs and load balancers; release the unattached ones.
- Compare GPU instance hours against the useful hours you logged. The gap is your utilisation problem.
- Note egress by destination. If inter-region transfer appears, ask why the data is in two regions.
- Set a billing alert at the level you expect next month, so the next surprise is a notification rather than an invoice.
None of this changes your hourly rate. All of it changes your bill.