Self-Hosted LLM vs API: The Break-Even, Calculated

The whole debate reduces to one line:

Break-even tokens per month = monthly GPU cost ÷ API price per token.

That formula is correct and almost always produces the wrong answer, because it’s missing a constraint. Here it is with the missing piece:

…and then check the result against the card’s physical ceiling, because that’s usually where the plan dies.

A GPU can only emit so many tokens a second. Multiply that by the hours in a month and you have a hard maximum. If your break-even volume is above that number, there is no volume at which self-hosting wins — not “you’d need to be very busy”, but arithmetically impossible.

Running the numbers

Take a realistic self-hosted setup for a 70B-class open-weight model: a rented A100 at $1.19/hour on RunPod’s community tier. Left running continuously that’s $868.70/month, and at roughly 20 tokens per second it can emit at most 52.6 million tokens a month.

That gives a floor cost of $16.53 per million tokens — the best you can possibly do, at 100% utilisation, with no idle time at all.

Now the break-even table:

Read the top row twice. Against a hosted version of the same model, self-hosting is 52× more expensive and there is no break-even at any volume — you would need fifty-two times more output than the card can physically produce.

This is not a marginal call that turns on your usage patterns. The gap is an order of magnitude wider than any assumption in the calculation.

Why hosted inference is so much cheaper

It isn’t magic, and understanding it tells you when the logic might not apply.

API providers run batched inference. Dozens of requests share a forward pass, so one GPU serves many users at once and the fixed cost of the hardware is amortised across all of them. Your self-hosted instance serves exactly one user — you — and pays for the whole card regardless.

They also buy hardware at a scale you don’t, run it at utilisation you can’t, and compete with each other on price.

So the 52× isn’t inefficiency on your part. It’s the structural difference between a shared resource and a dedicated one, and no amount of tuning closes it.

The gap widens further if you use the tools the APIs give you. Anthropic’s Batch API cuts input and output prices by 50%, and cache hits cost 10% of standard input. Both are free to adopt and both make self-hosting look worse.

The comparison people actually make, and why it’s wrong

Look at the bottom two rows again. There is a break-even against Claude Opus 5 and Fable 5 — 66% and 33% utilisation respectively, which is demanding but not impossible.

That comparison is invalid, and it’s the one most “self-host and save” arguments quietly rely on.

A self-hosted Llama 70B is not a substitute for Opus 5. They are different models with different capabilities, and swapping one for the other is a product decision, not a cost optimisation. If a 70B open-weight model does your job, then the honest comparison is against a hosted 70B at $0.32 — where self-hosting loses by 52×. If it doesn’t do your job, the self-hosted option isn’t on the table at any price.

The valid version of the frontier comparison is: “could I switch from Opus to a 70B model and accept the quality change?” If yes, switch — and then use a hosted 70B, which is cheaper than both.

What utilisation does

Everything above assumed 100% utilisation, which nobody achieves. Real self-hosters run somewhere between 5% and 15% — a few interactive sessions a day, some batch work, long idle stretches.

At 10% utilisation, that $16.53 per million becomes $165 per million. Against DeepInfra’s $0.32 you are now paying over 500 times the market rate.

Utilisation is the multiplier that turns a bad deal into an absurd one. It’s also the reason hourly billing matters so much: if you must self-host, rent by the hour and turn it off, rather than paying for a card that idles 90% of the time.

The three reasons that do justify it

None of them are cost. All of them are real.

1. Data residency and privacy. If your data legally or contractually cannot go to a third party, price isn’t the criterion — the API isn’t an option at all. This is the most common legitimate reason and it needs no cost justification.

2. No per-token ceiling. A self-hosted model has a fixed monthly cost and unmetered output. For workloads where usage could spike unpredictably — and where a surprise API bill would be worse than a predictable large one — that ceiling has genuine value. You’re buying budget certainty, not cheapness.

3. Deterministic latency and availability. No rate limits, no queueing behind other customers, no provider deprecating your model with 60 days’ notice. If your product breaks when inference is slow or a model version disappears, controlling the stack is worth paying for.

Notice what these have in common: each is a property you cannot buy from an API at any price. That’s the correct test for self-hosting. If you can articulate which of the three applies to you, self-host with confidence. If you can’t, you’re about to pay 50× for nothing.

If you’re self-hosting anyway

RunPod is the sensible default — hourly billing on community and secure tiers, published rates, and the ability to stop paying the moment you stop generating. The A100 rate used throughout this article is its community-tier price. Hourly billing is not a nicety here; it’s the only thing standing between you and the $868/month figure.

DigitalOcean offers GPU droplets with predictable billing and its usual documentation quality, which suits teams who want inference alongside existing infrastructure. Its tracked link isn’t working on our end, so we’ve named it without linking.

Hetzner sells dedicated GPU servers monthly rather than hourly, which is the right shape only if your utilisation is genuinely high — we compared the two directly in RunPod vs Hetzner for Ollama, where the answer turns entirely on duty cycle. Note also that Hetzner repriced its cloud line sharply in June 2026.

If you’re evaluating a smaller model on cheaper hardware, Ollama VPS cost runs the same arithmetic at the 7–8B scale, including the CPU-only option and the three cases where it’s rational. And if you want the hardware comparison rather than the economics, GPU VPS for LLM covers the cards.

How we checked this

GPU rates are RunPod’s published community-tier pricing in August 2026 — A100 PCIe at $1.19/hour, with an H100 PCIe at $1.99 and an RTX 4090 at $0.34 for reference.

API prices are the providers’ published rates: DeepInfra’s Llama 3.3 70B at $0.10 input and $0.32 output per million tokens, Together’s 70B at $0.88, and Anthropic’s Claude pricing of $1/$5 for Haiku 4.5, $2/$10 for Sonnet 5, $5/$25 for Opus 5 and $10/$50 for Fable 5 per million input/output tokens, along with the Batch API’s 50% reduction and cache hits at 10% of standard input. Comparisons use output pricing, which is the higher and more representative figure for generation workloads.

The throughput assumption is the weak point, and it is doing a lot of work. We have used 20 tokens per second for a 70B-class model on an A100, and we have not measured it. Real throughput depends on quantisation, batch size, context length, the serving stack and the specific card. We’ve shown 15 and 25 tokens per second alongside it so you can see the sensitivity: the floor cost moves between $13.22 and $22.04 per million.

That sensitivity does not threaten the conclusion. The gap against a like-for-like hosted model is 52×. You would need throughput to be wrong by more than an order of magnitude — 1,000+ tokens per second on a single A100 — for the top row of that table to change. It isn’t.

Where the numbers would genuinely shift: if you serve many concurrent users, you get batching benefits too, which is the one scenario where self-hosted economics improve substantially. This article prices the single-user or small-team case. If you’re serving hundreds of concurrent requests, run the calculation again with your real batch size — you’ll still probably lose, but by much less.

What we did not do: no benchmarks, no measured tokens per second, no test of any provider’s delivered throughput. This is a cost model built on published prices and a stated throughput assumption.

The hosting links above are affiliate links. This article’s conclusion is that most readers should use an API, which earns us nothing.

FAQ

Is self-hosting an LLM cheaper than using an API?

Almost never. Against a hosted version of the same open-weight model, self-hosting on a rented A100 costs roughly 52× more per token, and there is no monthly volume at which that reverses — the card cannot physically generate enough output.

What is the break-even point for self-hosting an LLM?

Break-even tokens per month equals monthly GPU cost divided by API price per token — then compare that to the card’s maximum output. Against open-weight APIs the break-even sits above what the hardware can produce, so it doesn’t exist.

Why is API inference so much cheaper than self-hosting?

Providers batch many requests through one forward pass, so a single GPU serves many users and its cost is spread across all of them. A self-hosted instance serves one user and pays for the whole card, idle or not.

Does self-hosting get cheaper at high volume?

Only up to the card’s ceiling, which is reached long before break-even against a comparable hosted model. Serving many concurrent users helps, because you get some of the same batching benefit — run the numbers with your real concurrency.

When should I self-host an LLM?

When data cannot leave your infrastructure, when you need a fixed cost ceiling rather than per-token billing, or when you need deterministic latency and control over model versions. Each is something an API cannot sell you at any price.

Isn’t a self-hosted model cheaper than Claude or GPT?

Per token, sometimes — but that compares different capabilities. If a 70B open-weight model does your job, compare it against a hosted 70B, which beats self-hosting by roughly 52×. The model choice and the hosting choice are separate decisions.

What utilisation do self-hosters actually achieve?

Typically 5–15%. At 10%, a floor cost of $16.53 per million tokens becomes $165 per million — over 500 times what a hosted equivalent charges.