A single GPT-5-class prompt at scale costs $0.30 to $2.50 per million tokens on the API; running the same workload on your own H100 lands around $0.12 per million output tokens. The crossover where self-hosting actually wins sits near 800 million tokens per month — and it falls roughly 35% year-over-year as neocloud H100 prices drop and serving software gets more efficient. We mapped list prices, prompt-cache multipliers, batch discounts, and self-host capex across twelve 2026 providers. Here is the actual unit economics, with the math.
The 2026 inference stack: 12 providers, four cost tiers
The 2026 inference market sorts cleanly into four cost tiers, and the gap between them is wide enough that the tier you pick often matters more than the model you pick within a tier.
For the sister unit-economics question on retrieval infrastructure, our vector database benchmark 2026 walks the same framework for pgvector, Pinecone, and Qdrant — the cost-per-query framing generalizes beyond LLMs.
Tier 1 — Closed flagship. OpenAI’s GPT-5.6 Sol, Anthropic’s Claude Opus 4.8, and Google’s Gemini 3.5 Pro lead the reasoning-frontier category. List prices sit in the $3 to $5 per million input tokens range and $15 to $30 per million output tokens. The capability-per-dollar is highest at this tier, but the raw unit cost is also the highest — output tokens are where the bill actually lives, because most agents and RAG pipelines emit 3-10x more output than input.
Tier 2 — Closed mid-tier and open-frontier gateways. Anthropic’s Claude Sonnet 4.6 at $3 input / $15 output, Google Gemini 2.5 Flash at $0.30 input / $2.50 output, Mistral Large 3 at $0.50 input / $1.50 output, and the gateway-hosted open-weights frontier (Together, Groq, Fireworks, DeepSeek, xAI Grok) all cluster in the $0.50 to $2.50 output range. Most production workloads that don’t strictly require closed-flagship reasoning land here — and they should, because the capability gap between Sonnet 4.6 and Opus 4.8 is much smaller than the cost gap.
Tier 3 — Cheap non-reasoning chat. Gemini 2.0 Flash-Lite at $0.075 input / $0.30 output, xAI’s Grok Build 0.1 at $1.00 / $2.00, and GPT-5.6 Luna at $0.20 / $1.20 occupy the bottom tier. Useful for classification, extraction, routing, and bulk reformatting where you don’t need the model’s best thinking.
Tier 4 — Self-hosted open weights. Llama 4 70B, Qwen 3, Mistral Large 3, DeepSeek V3, and the small-model family (Llama 3.x 8B, Qwen 3 4B) on rented or owned GPUs. At 85%+ utilization with FP8 quantization on an H100, published benchmarks put Llama 4 70B at $0.12 to $0.18 per million output tokens — well below any API in Tiers 1-3 for the same capability class.
The four tiers are not equally attractive at every monthly volume. We will unpack each below, then layer in caching and batch discounts, then run the self-host break-even math. The numbers cited throughout come from vendor pricing pages retrieved in early August 2026 unless otherwise noted.
Closed-flagship pricing: GPT-5.6, Claude Opus 4.8, Gemini 3.5
OpenAI’s published API rates for the GPT-5.6 family are the headline numbers: GPT-5.6 Sol at $5.00 input and $30.00 output per million tokens, GPT-5.6 Terra at $2.00 / $12.00, GPT-5.6 Luna at $0.20 / $1.20. Cached input runs at 10% of the standard rate, so a Sol-tier prompt with a 5,000-token system prefix that you re-use across 100,000 requests bills the cache at $0.50 per million rather than $5.00 — a 90% discount on the long-prefix portion. OpenAI’s Batch API adds another 50% off if you can tolerate 24-hour processing.
The current GPT-5.x line also exposes “Fast mode” (the rebrand of priority processing), which doubles the standard token cost in exchange for guaranteed latency. If you have a tight SLO and can’t tolerate tail-latency spikes, you are paying twice as much per token as the published headline rate. For most batch and asynchronous pipelines, leave it off.
Anthropic’s pricing is comparable at the top end. Claude Opus 4.8 lists at $5.00 input and $25.00 output per million tokens; Claude Sonnet 4.6 at $3.00 / $15.00; Claude Haiku 4.5 at $1.00 / $5.00. The interesting wrinkle is Claude Sonnet 5’s introductory pricing window: through August 31, 2026, Sonnet 5 runs at $2.00 / $10.00, then reverts to $3.00 / $15.00 from September 1. If you have a workload that fits Sonnet 5’s capability profile, the next four weeks are the cheapest Sonnet 5 will ever be. For a worked example of capability-vs-cost at the flagship tier, our Claude vs GPT-5 code review benchmark walks through per-task unit economics on a real engineering workflow.
Anthropic’s prompt caching is more aggressive than OpenAI’s: cache hits discount to 10% of input price (90% off), and cache writes cost 1.25x input for a 5-minute TTL or 2x input for a 1-hour TTL. The 1-hour TTL is the right pick for any agent that has a multi-turn conversation with a stable system prompt — the amortized cost is roughly half of the published headline because you only pay the 2x write premium once per hour, not per request.
Google’s flagship pricing tells a different story. Gemini 3.5 Flash is $1.50 input and $9.00 output per million tokens, which puts it between Anthropic’s Sonnet 4.6 and Google’s own older Gemini 2.5 Flash. Gemini 2.5 Flash at $0.30 / $2.50 is the real workhorse tier — fast enough for production, cheap enough that you stop measuring individual requests, and benchmarked within a few points of Sonnet 4.6 on most reasoning tasks that don’t require tool-use gymnastics.
The under-discussed Gemini tier is 2.0 Flash-Lite at $0.075 input and $0.30 output per million tokens — the cheapest published rate in any major provider’s catalog. It is not a reasoning model, it is not great at multi-step tool use, but for classification, extraction, reformatting, embeddings-adjacent work, and bulk routing it is genuinely competitive. If you are paying $5/M input for a GPT-5.6 Sol to do classification, you are leaving a 67x margin on the table.
Open-weights on someone else’s hardware: Together, Groq, Fireworks, DeepSeek
The mid-2026 frontier for cost-per-token sits with open-weights models hosted on infrastructure-specialist gateways. Together AI publishes Llama 3.3 70B at $0.88 input / $0.88 output per million tokens. Groq’s LPU-hosted Llama 3.3 70B runs at $0.59 / $0.79 per million tokens. Fireworks lists the same model at $0.90 / $0.90. These are competitive on price against Gemini 2.5 Flash and dramatically cheaper than any closed-flagship option for capability-equivalent 70B-class models. For the narrower question of when small models win on capability, our small language models 2026 piece walks through the 7B-vs-70B economics from the deployment side.
The trade-off is latency and capability ceiling. Groq’s LPU engine is the fastest published token-stream latency on Llama 3.3 70B in the 2026 market, which makes it the right pick for real-time voice agents and interactive chat where every millisecond matters. Together’s edge is flexibility — they host the broadest catalog of open-weights models with consistent pricing, and their batch tier drops further to $0.65/M tokens for 24-hour processing of larger models like Llama 4 70B. Fireworks sits between the two on price and offers deeper function-calling infrastructure.
DeepSeek is its own tier. DeepSeek R1 V3.2 runs at $0.28 input and $0.42 output per million tokens for the general model, with cache hits at $0.14/M. The standard R1 reasoning model — which charges output for all chain-of-thought tokens, not just the final answer — runs at $0.55 input and $2.19 output. The Nature paper documenting the training cost (about $294,000 for 512 H800 GPUs over 80 hours) explains why the price is so low: DeepSeek’s training efficiency has translated directly into inference efficiency. The widely-cited “$6 million training cost” figure refers to a specific narrow snapshot of the V3 base-model hardware utilization, not the R1 final cost — including failed experiments and R&D, total investment is closer to $500M to $1B per independent analysts. For the upstream reasoning-model analysis, our OpenAI o3 piece covers what reasoning models actually do differently with token budgets — the chain-of-thought cost surcharge is the same architectural choice as o3’s longer thinking steps.
xAI’s Grok is the wildcard. Grok 4.5 runs at $2.00 input and $6.00 output per million tokens with cache hits at $0.50, while Grok 4.3 and the Grok 4.20 family sit at $1.25 / $2.50. Long-context requests (over 200,000 prompt tokens) are billed at 2x the standard rate — a non-trivial cost for RAG workloads that pull large context windows. The Grok pricing is competitive for capability, but the long-context surcharge is the kind of detail that wrecks budgets if you do not read the footnotes.
Self-hosting: the formula and the break-even curve
Self-hosting wins on raw unit cost past a threshold, but the threshold is non-obvious. The packet.ai benchmark on Llama 4 70B at FP8 quantization, batch size 8, on a single H100 at 85%+ utilization, reports $0.12 to $0.18 per million output tokens. Compare that to GPT-5.6 Sol at $30 per million output tokens — a 165-250x gap on paper. Reality is messier: production throughput is typically 30-50% lower than peak lab figures due to context-length variance and tail-latency constraints, and you have to pay for engineers, monitoring, and the GPU you are not using during quiet hours.
The canonical cost formula is:
Cost per 1M output tokens = (GPU hourly rate × GPU count) /
(tokens per second × 3600 × utilization / 1e6)Plugging in the H100 mid-market rental of $2.95 to $3.46 per hour, TensorRT-LLM throughput on Llama 3.3 70B FP8 at batch size 8 of approximately 2,100 tokens per second at 50 concurrent requests, and 85% utilization, you land at $0.45/M output tokens — closer to the API than the lab benchmark suggests, because utilization rarely holds at 85% in production. Drop concurrency to 10, throughput to ~700 tokens per second, and you are paying $2.20/M output tokens — which loses to Groq at $0.79/M.
The hardware tier matters more than the hourly rate. Nvidia Blackwell (B200) deployments deliver approximately 10x cost reduction per token compared to H100 for batched inference workloads, putting well-tuned B200 self-host well below $0.05/M output tokens at scale. That is the destination the frontier is heading toward — but until B200 capacity is broadly available on neoclouds (mid-2026 is still mostly H100/H200), H100 self-host is the practical baseline.
The neocloud rental spread is wide enough to move your bill materially. RunPod’s Community Cloud H100 PCIe is $1.99 per hour, Modal is $3.95/hr, Lambda’s on-demand H100 PCIe is $3.29/hr and 8x H100 SXM is $3.99/GPU-hr. Spot and marketplace listings can fall below $1.50/hr with interruption risk, and reserved capacity on a one-year commit brings the effective hourly rate down further. The single biggest lever in self-hosting unit economics is not model selection or quantization — it is which provider you rent from.
Consumer and prosumer hardware is a different curve. The Mac Studio M3 Ultra (96 GB unified memory) runs 70B-class models comfortably without aggressive quantization, while the RTX 4090 (24 GB VRAM, 1008 GB/s bandwidth) is the fastest single consumer GPU for models up to 34B. Amortize a $5,000 Mac Studio over 24 months at moderate utilization and the effective hourly rate lands around $1.04 — competitive with the cheapest neocloud H100, with the bonus of zero queue time and zero data leaving your network. For the practical setup walkthrough, our local LLM setup 2026 piece covers Ollama, LM Studio, and GPT4All against the same hardware matrix. The runtime-cost picture is incomplete without the energy side — our AI energy consumption 2026 piece maps the per-query wattage and grid-side cost that the GPU-hour number alone leaves out.
The crossover: self-hosting wins at roughly 800M tokens per month for a 70B-class model against GPT-5.6-class APIs, and it falls about 35% year-over-year as neocloud H100 prices drop and serving frameworks (vLLM, TensorRT-LLM, SGLang) get more efficient. Below that volume, the API is cheaper once you account for engineering hours. Above that volume, self-hosting wins — and the gap widens fast as you scale.
Effective cost: prompt caching, batch mode, and the stacking rules
The headline rates are misleading. Real workloads — agents, RAG pipelines, multi-turn chat — run on systems where the same long prefix gets re-sent hundreds of times per minute. Prompt caching is the single largest cost lever you control, and the rules differ by vendor.
OpenAI, Anthropic, and Google all offer a 50% batch discount for 24-hour processing. Stacking rules:
- Anthropic: Batch discount and caching multiplier stack explicitly as stated on their pricing page.
- OpenAI: Stacking is supported for GPT-5 and later. For older models, use the Flex processing path to combine the 50% discount with full prompt caching.
- Google: Explicit context caching is supported with published discounted batch rates. Implicit (automatic) caching inside batch jobs remains undocumented.
The compounding effect on a long-prefix agent workload is dramatic. A 4,000-token system prompt re-sent 1,000 times per minute for 8 hours a day costs roughly: without caching, 4,000 × 1,000 × 60 × 8 × 30 = 57.6 billion input tokens per month, at $5/M = $288,000. With prompt caching at 10% of input price (cache write once per TTL window), the same workload drops to about $30,000. With batch mode on top, $15,000. Caching plus batch can drop effective cost by 90-95% on workloads with stable prefixes — far larger than the savings from picking a cheaper model. For the technical depth behind long-prefix economics and how context windows actually behave, our LLM context windows 2026 piece maps the same problem from the engineering-reality side.
The catch: cache writes themselves cost more than fresh tokens. Anthropic’s 1-hour TTL cache write is 2x input price, so you only win if your cache hit rate is high enough to amortize the write premium. For a 4,000-token prefix re-sent 1,000 times per hour, the break-even is roughly 50% — well below the 90%+ hit rate a long-system-prompt agent actually achieves.
The cost curve: 10x/year decline, 3.4x/year demand growth
The 2026 inference market is shaped by two compounding curves. Per-token cost is falling 9x to 900x per year depending on capability benchmark, with a typical ~10x annual decline. Drivers are mostly software and architectural: Mixture-of-Experts routing, speculative decoding, FlashAttention, paged attention, FP8 and FP4 quantization. Hardware matters — Blackwell delivers another 10x — but the software curve was already delivering the bulk of the savings before Blackwell shipped. For the underlying hardware roadmap context, our AI hardware race 2026 analysis covers the chip-side story in detail.
Token demand is rising about 3.4x per year, driven by agentic workloads and long-context tasks. The net effect is that absolute inference spend is climbing even as per-token cost plummets. A team that grew their token volume 3x last year at flat budget did not save money — they ran twice as hot on the absolute-spend line because cost-per-token only declined ~3x while volume grew ~3.4x.
This is the analytical core of the post. The “inference is cheap now” framing is true on a per-token basis and misleading in absolute terms. If you are an engineering lead planning capacity for 2027, the right unit of planning is not “what does one prompt cost?” but “what is my token-growth rate, and how does that interact with the cost curve?” A team whose tokens double every six months is in a different procurement posture than a team whose tokens are flat — even if both teams currently pay the same per-token rate.
The trajectory also implies that self-host break-even crossovers are not stable. The 800M tokens/month figure is a moving target — it falls roughly 35% per year as neocloud H100 prices drop and serving efficiency improves. A workload that does not justify self-hosting today may justify it next year, and a workload that just barely justifies it today will be a runaway win by 2027.
What to do this week: a procurement checklist
You can map your real workload to this framework in four mechanical steps.
- Audit your last 30 days of token usage by model and prompt shape. Most teams discover that 70-90% of their bill is concentrated in 2-3 prompt shapes — usually a long-prefix agent loop or a RAG re-rank step. Those are the workloads where caching, batching, and tier-down pay off the most.
- Compute your actual cache-hit ratio on the long-prefix workloads. If it is below 50%, your caching is misconfigured — the default Anthropic and OpenAI cache TTLs are 5 minutes; the 1-hour Anthropic TTL is usually the right pick for agent loops. If you are above 80%, you are already getting most of the available savings.
- Run a two-week self-host POC at your real concurrency. Pick one workload, deploy it on Modal or Lambda at your actual request volume (not synthetic), measure tokens-per-second and hourly GPU cost, and compute the real cost-per-million-token including engineering overhead. Most teams learn whether self-hosting wins within two weeks of data.
- Compare your monthly token-growth rate against Epoch’s 3.4x/year baseline. If your growth is below 1.5x/year, the API is the right answer and you are optimizing the wrong variable. If your growth is above 5x/year, self-hosting is a question of when, not if — and the sooner you POC, the better positioned you are when the crossover arrives.
One concrete rule of thumb: if your monthly bill crossed $20,000 at the GPT-5.6 Sol tier and your token growth is non-trivial, you have already crossed the break-even for self-hosting at least one workload. Pick the biggest one and POC it. The unit economics are not subtle — they are arithmetic. The only failure mode is not running the math.
Frequently asked questions
How much does GPT-5 cost per million tokens in 2026?
OpenAI’s GPT-5.6 Sol lists at $5.00 input and $30.00 output per 1M tokens; GPT-5.6 Terra at $2.00 input and $12.00 output; GPT-5.6 Luna at $0.20 input and $1.20 output. Cached input runs at 10% of standard, and the Batch API cuts that in half again if you can tolerate a 24-hour processing window.
Is self-hosting an LLM actually cheaper than the API in 2026?
It depends entirely on monthly volume. At roughly 85% utilization with FP8 quantization on an H100, self-hosted Llama 4 70B lands around $0.12 to $0.18 per million output tokens — well below the $30/M output rate for GPT-5.6 Sol. The break-even crossover against a closed API sits near 800M tokens per month, and it falls roughly 35% year-over-year as neocloud H100 prices drop and serving software gets more efficient.
What’s the cheapest LLM API in 2026?
For flagship-tier reasoning, DeepSeek R1 V3.2 at $0.28 input and $0.42 output per 1M tokens is the published floor among major providers. For non-reasoning chat, Gemini 2.0 Flash-Lite at $0.075 input and $0.30 output is cheaper still, but the capability gap to GPT-5-class models is real. The cheapest route through a gateway depends on the model: Groq publishes Llama 3.3 70B at $0.59 / $0.79 per million tokens, the lowest among major 70B gateways.
Why do inference costs keep dropping so fast?
Epoch AI’s published analysis puts the rate at 9x to 900x per year depending on which capability benchmark you anchor to, with a typical ~10x annual decline. The drivers are mostly software and architectural rather than hardware alone: Mixture-of-Experts routing, speculative decoding, FlashAttention, paged attention, FP8 and FP4 quantization, plus generational hardware gains from Hopper to Blackwell. A single B200 deployment can deliver ~10x cost reduction per token compared to H100 on batched workloads, but the software-side curve was already delivering most of the savings before Blackwell shipped.