The Best GPU Hosting for vLLM in 2026
The most common vLLM mistake is buying the GPU before doing the arithmetic. vLLM’s memory requirement is not “the size of the model” — it’s the weights plus the KV cache, and the KV cache grows with context length and how many requests you serve at once. A card that looks comfortable for an 8B model at 4k context can fail to start the same model at 32k.
There is also a vLLM-specific detail that catches people: gpu_memory_utilization defaults to 0.9. vLLM pre-allocates roughly 90% of the card up front, weights and KV cache together, rather than growing into it. That’s deliberate — the pre-allocation is what makes PagedAttention and continuous batching fast — but it means “the model is only 16GB, so 24GB is fine” isn’t how the budget works.
Before the provider list, three things worth knowing:
- VRAM is the hard constraint. A faster GPU cannot load a model that doesn’t fit. Architecture matters second.
- Utilisation decides the bill, not the hourly rate. An idle H100 is the most expensive thing in your stack.
- A hosted API is usually cheaper until the accelerator is genuinely busy. That crossover is further out than most people assume.
Quick comparison
| Provider | Billing shape | Scale to zero | Best for |
|---|---|---|---|
| RunPod | Per second, Pods + Serverless | ✅ Flex workers | Whole lifecycle, one provider |
| Modal | Per second | ✅ No idle charge | Bursty Python inference |
| Lambda | Per GPU hour | ❌ Bills to termination | Continuously busy endpoints |
| Koyeb | Per hour | ✅ Preview | PaaS-style deployment |
| Vast.ai | Marketplace, variable | ❌ | Cheap batch and fine-tuning |
| CoreWeave | On-demand, spot, committed | Depends | Enterprise fleets |
| Replicate | Per prediction | ✅ Managed | Skipping GPU ops entirely |
| DigitalOcean | Per hour | ❌ Powering off stops compute | GPU Droplets and notebooks |
Representative public prices below were checked in mid-2026 and move constantly, especially on marketplaces. Use them for billing shape, then price your actual deployment.
The sizing arithmetic
Two numbers, added together.
Weights. Roughly 2 bytes per parameter at FP16/BF16, or about 0.5–0.6 bytes at 4-bit quantisation. So an 8B model is ~16GB at half precision and ~5GB quantised. A 70B model is ~140GB at FP16 — which is why 70B on a single 80GB card requires quantisation.
KV cache. This is the part people forget. Every token in every active sequence holds attention state, so the cache scales with context length × concurrent requests. Serving one user at 4k context is cheap; serving forty at 32k is not. On long-context or high-concurrency deployments, the cache can rival the weights.
Then vLLM claims 90% of the card for both, by default.
| Model | Precision | Weights | Sensible minimum card |
|---|---|---|---|
| 7–8B | 4-bit | ~5GB | 16–24GB |
| 7–8B | FP16 | ~16GB | 24GB (short context) / 40–48GB |
| 13B | 4-bit | ~8GB | 24GB |
| 13B | FP16 | ~26GB | 40–48GB |
| 30–34B | 4-bit | ~18GB | 40–48GB |
| 70B | 4-bit | ~40GB | 80GB |
| 70B | FP16 | ~140GB | Multiple 80GB cards |
The VRAM tiers map to workloads like this:
| VRAM | Fits | Watch out for |
|---|---|---|
| 16–24GB | Small quantised models, development | Little headroom for long context or batching |
| 40–48GB | 13B–30B quantised, real batching | Fit depends on precision and context |
| 80GB | Higher throughput, larger quantised models | 70B may still need quantisation or sharding |
| 96–180GB | Large models, long context, multimodal | Premium hardware wasted if traffic can’t keep it busy |
If you take one operational tip from this article: if vLLM fails to start with an out-of-memory error, don’t immediately buy a bigger card. Lower gpu_memory_utilization, shorten max_model_len, or reduce max_num_seqs first. Long context is usually the culprit, and it’s a configuration value rather than a hardware purchase.
The 8 best GPU hosts for vLLM
1. RunPod — best overall for vLLM
RunPod wins on covering the whole lifecycle without changing provider. You can start on an interactive Pod while you get the model serving, bake the result into a container, attach persistent model storage, and promote it to a Serverless endpoint — same account, same images, same volumes.
It also has one-click templates for vLLM specifically, which removes the fiddly part of the first deployment. And it offers a wider spread of consumer and data-centre cards than the enterprise-only platforms, so a 24GB card for a small quantised model is actually available rather than being forced onto an 80GB minimum.
Billing is per second, from around $0.00016/second on 16GB GPUs and roughly $0.00076/second for an A100 80GB.
The trap to understand before you rely on it: serverless does not remove infrastructure decisions, and RunPod bills the worker while it starts, loads the model, serves the request, and then waits out the configured idle timeout. A 30-second inference with a 90-second cold start is not a 30-second bill. Baked images, cached weights and network volumes all help, and each has its own cost and portability trade-off.
Pros: per-second billing; Pods and Serverless on one platform; vLLM templates; widest GPU selection; flex workers scale to zero; persistent network volumes.
Cons: cold starts are billed and can dominate short requests; storage billed separately; network volumes can constrain region choice.
Who it’s for: most vLLM deployments, and anyone whose traffic is spiky rather than constant.
Check RunPod’s current pricing →
2. Modal — best serverless for bursty inference
Modal is code-first: you define the deployment in Python and it handles autoscaling and scale-to-zero, with no compute charge while idle. For a team already writing Python — which is most teams serving vLLM — that removes a whole layer of platform work.
Representative rates run about $0.000164/second for a T4, $0.000542 for an L40S and $0.001097 for an H100.
The deployment model is opinionated. If you want a conventional VM you can SSH into, this isn’t it, and volumes and model-loading strategy need designing around startup time.
Pros: genuine scale-to-zero with no idle compute charge; Python-native deployment; strong autoscaling; per-second billing.
Cons: opinionated, Python-first model; less conventional-VM control; startup design matters for latency.
Who it’s for: bursty API-shaped inference where the team is comfortable deploying from Python.
3. Lambda — best for continuously busy endpoints
Lambda is the pick when the GPU stays busy. Dedicated instances bill until you terminate them, so there is no scale-to-zero — but the hourly rates are competitive, and above a certain utilisation dedicated beats serverless outright.
Representative pricing: A6000 from $1.09/GPU-hour, H100 PCIe from $3.29, H100 SXM from $4.29.
For an always-on vLLM endpoint with steady traffic, this is a strong default — you’re paying for a machine and using all of it. For anything intermittent it’s the wrong shape.
Pros: competitive dedicated hourly rates; predictable access to enterprise cards; persistent file systems; simple mental model.
Cons: no scale-to-zero, billing continues until termination; poor fit for spiky traffic; local instance data shouldn’t be treated as durable.
Who it’s for: production endpoints with steady, high utilisation.
4. Koyeb — best PaaS-style deployment
Koyeb sits between a raw GPU host and a managed platform: you deploy a service and it handles the platform layer, with GPU scale-to-zero available in preview. Pricing is straightforward — A6000 at $0.75/hour, L40S at $1.20, H100 at $2.50 — and notably cheap at the mid-range.
For an always-on vLLM endpoint where you want platform conveniences without assembling them, it’s a strong and underrated option.
Scale-to-zero being in preview is the caveat; treat it as a roadmap item rather than a guarantee, and plan persistence around the service architecture.
Pros: competitive mid-range pricing; PaaS conveniences; scale-to-zero in preview; simple deployment model.
Cons: smaller platform than the majors; scale-to-zero not yet mature; persistence needs planning.
Who it’s for: always-on endpoints that want a platform rather than a server.
5. Vast.ai — cheapest, for people who can handle a marketplace
Vast.ai is a marketplace: independent hosts list capacity and you rent it, so prices sit well below the managed platforms and vary continuously by host, region and reliability rating.
For fine-tuning runs, batch jobs and checkpointed work that can tolerate interruption, the savings are real and substantial.
The trade-offs are exactly what a marketplace implies — variable reliability, variable performance, storage and both directions of transfer potentially billed separately, and no single support relationship. Do not put a synchronous production API on interruptible marketplace capacity.
Pros: by far the cheapest compute; wide hardware selection; good for interruptible batch work.
Cons: variable reliability and performance; separate storage and transfer billing; unsuitable for synchronous production serving.
Who it’s for: experienced users doing fine-tuning or batch inference on a budget.
6. CoreWeave — best for enterprise fleets
CoreWeave is built for organisations running many models across a lot of hardware: managed Kubernetes, several storage architectures, serious networking, and committed-capacity pricing alongside on-demand and spot.
If you’re serving one vLLM endpoint this is more platform than you need. If you’re running a fleet with capacity planning and multi-model serving, it’s the one on this list designed for that.
Pros: enterprise-grade Kubernetes, storage and networking; committed-capacity options; built for scale.
Cons: substantial overkill for a single endpoint; pricing varies by GPU, region and commitment; assumes platform expertise.
Who it’s for: larger teams running multi-model inference at scale.
7. Replicate — the managed alternative to hosting at all
Replicate runs the model for you and charges per prediction. It isn’t GPU hosting in the same sense, which is exactly why it belongs here: for many people it’s the correct answer, and choosing it means never debugging a CUDA version.
Public models charge for active processing; dedicated deployments also charge setup and idle time. Model hosting is managed, so there’s no storage administration.
You give up low-level control — quantisation settings, batching parameters, serving flags. If you chose vLLM specifically to tune those, this defeats the purpose.
Pros: no GPU administration; per-prediction billing; fast to deploy known models; managed storage.
Cons: little control over serving parameters; dedicated deployments bill idle time; less economical at sustained high volume.
Who it’s for: prototyping, and teams who want inference without infrastructure.
8. DigitalOcean — GPU Droplets and Paperspace notebooks
DigitalOcean covers this two ways, and most comparisons still only mention the older one. GPU Droplets are conventional NVIDIA and AMD GPU instances you spin up like any other Droplet — the straightforward option if you want a normal VM running vLLM inside DigitalOcean’s platform. Paperspace, which DigitalOcean acquired, remains the notebook-and-machine workflow for model development.
For a vLLM endpoint, GPU Droplets are the relevant product: ordinary instances, ordinary networking, and the same console as the rest of your stack. Paperspace is the better fit for notebook-led experimentation.
Representative pricing: A6000 at $1.89/hour, A100 80GB at $3.18, H100 at $5.95 — noticeably above the specialists.
Powering off a machine stops compute billing, but attached storage and add-ons keep charging, which is a common source of surprise bills.
Pros: conventional GPU instances via GPU Droplets; notebook workflow via Paperspace; integrates with the wider DigitalOcean platform; NVIDIA and AMD options.
Cons: clearly the most expensive per GPU hour here; storage bills after shutdown; no scale-to-zero.
Who it’s for: teams already standardised on DigitalOcean, and notebook-led development.
Check DigitalOcean’s current pricing →
Serverless or always-on? Do the division
This section assumes you’ve settled on renting rather than buying. If you haven’t, our GPU VPS guide covers the procurement decision — hourly versus committed versus owning a card — and the utilization thresholds where each one wins.
There’s a clean formula for this, and it beats intuition:
break-even utilisation = always-on hourly price
÷ (serverless per-second price × 3,600)
Worked through: an always-on L40S at $1.20/hour against a serverless L40S at roughly $1.95/hour-equivalent means the dedicated instance wins above about 62% utilisation. Below that, scaling to zero saves money.
Real break-even sits lower than the arithmetic suggests once you include cold starts, any minimum warm workers you keep, failed requests, and the engineering time to operate an autoscaler.
Practical sequence: start serverless while demand is unknown, measure actual utilisation, and only move to dedicated once the GPU is genuinely busy. And before migrating because the bill looks large, check whether continuous batching, quantisation or a smaller card fixes utilisation without changing anything else.
When a hosted API is simply cheaper
This deserves stating plainly on a page that earns money when you rent a GPU.
For most workloads below serious sustained volume, a hosted model API is cheaper than self-hosting vLLM — no idle accelerator, no model downloads, no autoscaler, no evenings lost to CUDA versions. Current published list rates for Anthropic’s models are $1/$5 per million tokens for Haiku 4.5, $3/$15 for Sonnet 5 and $5/$25 for Opus 5. Against an H100 at $3.29–5.95/hour that idles between requests, the API wins comfortably until the card is busy.
Renting a GPU becomes the right call when at least one of these is true:
- Traffic is steady enough to keep the accelerator genuinely occupied
- The model uses private weights or custom code no API offers
- You need a dedicated warm endpoint for latency reasons
- Per-request API margins stop working at your scale
- You need low-level control over quantisation, batching or serving parameters
- Data-handling rules require a controlled environment
A pattern worth copying: split the architecture. Keep the application, auth, database and queues on ordinary infrastructure and send only inference to the GPU provider. That keeps the expensive hardware doing model work exclusively, and makes everything else cheaper to scale and easier to recover.
How we picked
We ranked on what determines the bill and the latency for a vLLM deployment specifically: billing granularity, whether the platform can scale to zero, cold-start behaviour, GPU selection at the VRAM tiers that matter for common model sizes, and persistent storage for model weights. Prices are representative public rates checked in mid-2026; marketplace pricing in particular moves continuously, so treat every figure as a starting point rather than a quote.
The sizing guidance is arithmetic — parameter counts, precision and KV-cache behaviour — rather than benchmark results, and vLLM’s own defaults are cited where they change the calculation. We hold affiliate relationships with two of the eight providers here; the other six earn us nothing, and the “an API is probably cheaper” section above earns us nothing at all.
How to choose
- Do the VRAM arithmetic first. Weights plus KV cache, then remember vLLM claims 90% of the card by default.
- Estimate utilisation honestly. Below roughly 60%, serverless. Above it, dedicated.
- Should you self-host at all? Compare against API list pricing before committing. For most projects the API wins.
- How spiky is traffic? Bursty favours RunPod or Modal. Steady favours Lambda or Koyeb.
- Can the work be interrupted? If yes, Vast.ai is dramatically cheaper.
For most people the answer is RunPod — per-second billing, vLLM templates, the widest card selection, and a path from experiment to production without switching provider. Choose Modal if you’d rather deploy from Python, Lambda or Koyeb for a genuinely always-on endpoint, Vast.ai for interruptible batch work, CoreWeave at fleet scale, DigitalOcean if your stack already lives there, and Replicate if the honest answer is that you don’t want to run a GPU at all.
FAQ
How much VRAM does vLLM need?
Weights plus KV cache, and vLLM pre-allocates about 90% of the card by default via gpu_memory_utilization. As a guide: an 8B model needs roughly 5GB quantised or 16GB at FP16, a 13B around 8GB or 26GB, and a 70B about 40GB quantised. Add headroom for the KV cache, which grows with context length and concurrent requests — long context is what usually pushes a deployment onto a bigger card.
Why does vLLM run out of memory on a GPU that should fit the model?
Almost always the KV cache or the default pre-allocation. vLLM reserves ~90% of VRAM for weights and cache together at startup, and the cache scales with max_model_len and max_num_seqs. Before buying a larger card, try lowering gpu_memory_utilization, reducing max_model_len, or cutting the number of concurrent sequences.
Is serverless GPU hosting cheaper than a dedicated instance?
Below about 60% utilisation, usually yes. Divide the always-on hourly rate by the serverless per-second rate times 3,600 to get your break-even. Cold starts, minimum warm workers and failed requests push the real crossover lower than the raw arithmetic, so start serverless when demand is unknown and move to dedicated once the GPU is measurably busy.
Is self-hosting vLLM cheaper than using an API?
Only once the GPU is genuinely busy. Hosted model APIs list at $1–5 per million input tokens depending on tier, with no idle cost, no model loading and no infrastructure to operate. A dedicated H100 costs $3.29–5.95 per hour whether or not requests arrive. Self-hosting wins on private weights, dedicated latency, low-level serving control, data-handling requirements, or sustained volume high enough that per-token pricing dominates.
Related Blog Posts
