Bottom line for the next 24 months
Bottom Line: Neither open-weights nor frontier APIs win universally — a three-layer portfolio beats both single-vendor API dependence and a premature full commitment to self-hosting. For the next 24 months, route capability-bound tasks (complex reasoning, long-context coding, agentic tool use) through frontier APIs such as OpenAI pricing tiers or Claude, run high-volume, cost-sensitive, or residency-constrained workloads on open-weight models (Llama 3.1 70B, DeepSeek-V3, Kimi K3) — either self-hosted or via a hosted open-weight API — and reserve bespoke fine-tunes for domain-critical tasks where neither layer hits quality thresholds. The break-even that determines how aggressively to shift spend toward open weights is driven by three variables: monthly token volume (self-hosted economics only favour open weights above roughly 500M–1B tokens/month depending on GPU utilisation), platform engineering FTE cost (typically 0.5–1.5 FTE for a self-hosted cluster), and the rate at which vendor pricing and capability drift during your planning window. A model gateway is a prerequisite for this strategy, not an optional add-on — switching costs compound quickly once prompts, eval harnesses, safety filters, and rate-limit logic are hard-coded to one provider.
OpenAI pricing currently shows multiple GPT-5.6 tiers spanning roughly $1 to $5 per 1M input tokens and $6 to $30 per 1M output tokens, which means two teams on the same vendor can still be operating on materially different cost curves depending on tier choice. Claude's pricing page states: "Introductory pricing of $2/$10 per million input/output tokens is in effect through August 31, 2026, after which the standard pricing of $3/$15 per million input/output tokens will take effect." That kind of scheduled change is exactly why single-vendor dependence gets expensive fast. The Artificial Analysis leaderboard compares intelligence, pricing, speed, and latency across models, and in 2026 the front of the quality table remains tightly clustered — enough to justify portfolio routing rather than a single-vendor commitment.
What changes the decision: price, capability, and vendor risk
The 24-month framing matters because three macro forces each move on a sub-year cycle: API pricing, model capability, and residency or procurement constraints.
| Dimension | Frontier APIs (OpenAI, Anthropic) | Hosted Open Weights (OpenRouter, etc.) | Self-Hosted Open Weights |
|---|---|---|---|
| Price drift risk | High — price changes announced with short notice | Medium — aggregator pricing changes independently | Low — compute cost is your cloud contract |
| Capability drift | Rapid upward — new model tiers released frequently | Moderate — open releases lag frontier by weeks to months | Moderate — you control the upgrade cycle |
| Data residency | Vendor-dependent; limited sovereign options | Depends on aggregator's infra | Full control; fits most residency mandates |
| Procurement risk | BAA/DPA timelines, rate limits, deprecation windows | Aggregator lock-in, uptime SLAs vary | Internal procurement only; slower iteration |
OpenAI pricing currently exposes multiple GPT-5.6 variants with input costs ranging from $1 to $5 per 1M tokens and output costs ranging from $6 to $30 per 1M tokens — a materially different economics profile inside one vendor. Anthropic's pricing page documents a scheduled date-based increase: "Introductory pricing of $2/$10 per million input/output tokens is in effect through August 31, 2026, after which the standard pricing of $3/$15 per million input/output tokens will take effect." These are not hypothetical risks — they are published, confirmed, and already inside most active planning windows.
As the Artificial Analysis leaderboard tracks intelligence, pricing, speed, and latency, the smart procurement posture is to treat price drift and capability drift as independent risk dimensions and plan for both to move during the 12–24 month horizon.
Why single-vendor dependence gets expensive fast
Every hard dependency on a single frontier provider exposes a product to four compounding risks simultaneously: price increases, model deprecation, rate-limit changes, and capability jumps that force prompt rewrites. Anthropic's announced pricing step-up from $2/$10 to $3/$15 after 31 August 2026 is one confirmed data point; the pattern repeats across the market. OpenAI's tiered pricing means a team that built against a higher-cost tier and later migrates to a lower-cost tier must still revalidate evals, handle any output behaviour differences, and update rate-limit handling — migration friction exists even within a single vendor.
The market has already converged on gateway abstraction as the primary mitigation. OpenRouter provides access to 400+ LLMs from OpenAI, Anthropic, Google, Meta, and others through a single API. Requesty describes its offering as "The AI Gateway for Production — 600+ models, real-time analytics, intelligent routing." These products exist because model risk, not just cost, is a first-class enterprise concern.
Pro Tip: Deploy a model gateway on day one, even if your current routing is 100% to one provider. The gateway preserves optionality by decoupling your application's API surface from any single vendor's schema. When a vendor raises prices, deprecates a model, or a competitor releases a capability jump, you switch in the routing layer rather than rewriting the application. Gateways do not eliminate migration work caused by prompt or tool-schema differences, but they eliminate the architectural rework that makes migration prohibitively expensive.
Where open-weights change the economics
Open-weight models change the per-token cost structure, but only once utilisation is high enough to absorb fixed operational costs. vLLM's PagedAttention architecture reduces KV-cache memory waste and enables continuous batching, which is the mechanism that improves throughput and makes self-hosted inference economically viable at scale. At sustained high traffic, a self-hosted Llama 3.1 70B or DeepSeek-V3 cluster can deliver marginal token costs well below any frontier API list price. At low or bursty traffic, that same cluster idles expensive GPU-hours, and the frontier API pays only for what it serves.
Watch Out: Per-token cost is not total cost of ownership for self-hosted LLMs. A self-hosted deployment must account for GPU infrastructure, networking, monitoring, orchestration (typically vLLM behind a load balancer), on-call engineering, incident response, and platform engineering FTE. A 2026 self-hosting analysis notes: "Self-hosting an LLM in 2026 is much easier than it was in 2024, but it is not free of engineering work" (AuraByt). Teams that budget only GPU cost routinely underestimate total spend by 40–60%.
Landscape: frontier APIs, hosted open weights, and self-hosted deployments
| Layer | Representative Options | Control | Latency | Compliance Fit | Ops Burden |
|---|---|---|---|---|---|
| Frontier APIs | OpenAI, Claude | Low — model is fully managed | Low latency; rate-limit constrained | Vendor DPA required; limited residency | Minimal; vendor manages infra |
| Hosted open weights | OpenRouter (400+ models), Mistral Large via API | Medium — model choice is yours; infra is theirs | Variable by provider | Depends on aggregator's data handling | Low infra; medium integration overhead |
| Self-hosted open weights | Llama 3.1 70B, DeepSeek-V3, Kimi K3 on H100 via vLLM | High — full data and model control | Tunable; no external rate limits | Strong residency and audit fit | High — GPU ops, monitoring, on-call, upgrades |
OpenAI pricing and Anthropic's Claude pricing anchor the frontier-API layer with published per-token costs. OpenRouter exemplifies the hosted open-weight middle layer: "Compare 400+ LLMs from OpenAI, Anthropic, Google, Meta and more — pricing, context length, and benchmarks side by side, all through one API." The self-hosted layer is anchored by vLLM, which positions itself as a production inference server for open-weight deployments and uses PagedAttention to increase throughput and GPU utilisation.
When frontier APIs still win
Frontier APIs remain the correct default when model quality is the binding constraint on product outcomes. The Artificial Analysis leaderboard still places frontier models at the top of the quality rankings in 2026, and the score gap between the best frontier model and the best publicly available open-weight model is still meaningful on complex reasoning, agentic tool use, and long-context retrieval tasks. GPT-5.6 and Claude sit in premium tiers — but if a few points of benchmark lift translates into measurable user outcomes, the spend is justified.
OpenAI's multi-tier pricing also makes a partial frontier commitment viable: route your highest-stakes completions to the most capable tier, mid-tier tasks to a lower-cost tier, and high-volume, lower-stakes calls to the cheapest tier — all within one vendor, before any open-weight layer is needed.
Pro Tip: Reserve frontier APIs specifically for capability-bound tasks — those where model failure has direct product cost (factual errors in customer-facing output, tool-call failures in agentic workflows, compliance drift in regulated text). For classification, summarisation, or extraction at volume, the quality gap versus a fine-tuned open-weight model is often small enough that moving to a cheaper layer is defensible.
When hosted open weights are the pragmatic middle ground
Hosted open-weight APIs — OpenRouter, Mistral's own API, and similar aggregators — offer the capability range of open models without the operational burden of running GPU clusters. They suit teams that need multi-model flexibility, want to avoid single-vendor coupling, but lack the platform engineering capacity for full self-hosting.
The tradeoff is that the ops burden shifts rather than disappears. Prompt management, version pinning, eval harnesses, observability, and routing logic become internal platform concerns even when the inference is hosted externally. Teams consistently underestimate this.
Watch Out: Hosted open-weight APIs are not infrastructure-free. When a model version changes on the aggregator, your evals may silently drift. When the aggregator's uptime drops, you have no GPU capacity to fall back on. Build version-pinned routing, automated eval runs on each model update, and a fallback path to a frontier API into your architecture before traffic scales — not after the first incident.
When full self-hosting is worth the operational burden
Full self-hosting on H100 clusters running vLLM justifies its overhead under two conditions: sustained high token volume and/or sovereignty requirements that frontier APIs cannot satisfy.
Production Note: The economic case for self-hosted LLMs materialises when workloads are predictably high-volume (roughly 500M+ tokens/month as a conservative floor, depending on GPU utilisation achieved), when data locality is legally mandated (HIPAA, GDPR Article 44 cross-border transfer restrictions, financial sector data residency rules), or when the organisation needs full audit control over model weights and inference logs. At that intersection, the amortised GPU cost per token on a well-utilised vLLM cluster running Llama 3.1 70B or DeepSeek-V3 can undercut frontier API list prices by a factor of 3–5× on marginal token cost — but this calculation only holds if the ops overhead described in the TCO section is fully funded.
Cost and ROI model for a CTO decision
The cost decision is not a single number — it is a function of token volume, utilisation, FTE cost, and time to break-even. The table below maps three deployment layers to realistic cost buckets using the pricing anchors captured in August 2026.
| Layer | Token cost (input/output per 1M) | Infra/month | Platform FTE overhead | Break-even vs. frontier API |
|---|---|---|---|---|
| Frontier API — premium tier | $3–$5 / $15–$30 | $0 | ~0.1 FTE (integration only) | Baseline |
| Frontier API — economy tier | $1 / $6 | $0 | ~0.1 FTE | Already cheaper; use by default for volume tasks |
| Hosted open weights | $0.10–$0.80 / $0.30–$2.50 (model-dependent) | $0 | ~0.3–0.5 FTE (routing, evals, version governance) | ~3–6 months to recover integration cost |
| Self-hosted open weights | ~$0.02–$0.08 / $0.02–$0.08 (at high utilisation) | $15k–$60k/month (8×H100 node range) | 0.5–1.5 FTE (ops, on-call, upgrades) | 12–18 months at 500M tokens/month; shorter at higher volume |
API spend bands by workload shape
OpenAI's pricing creates three natural spend bands: the cheapest tier suits high-volume, lower-stakes calls; the mid-tier suits mid-production tasks; the premium tier suits capability-bound workloads. Anthropic's Claude remains competitive by output-token price through 31 August 2026 before rising to its standard rate.
At low volume (under 50M tokens/month), API integration cost dominates and frontier APIs are nearly always the correct choice regardless of per-token price. At medium volume (50M–500M tokens/month), the economy-tier frontier APIs and hosted open weights are comparable once integration overhead is factored in. Above 500M tokens/month with predictable traffic, self-hosted open weights enter the break-even window — provided GPU utilisation exceeds 60–70%, which requires continuous batching via vLLM rather than naive sequential serving.
What self-hosting really costs beyond GPUs
GPU cost is the most visible line item and the most misleading one. An 8×H100 node runs $15k–$25k/month on major cloud providers at on-demand rates (reserved pricing reduces this 30–40%). But that number excludes:
- Networking and storage: NVLink fabric, high-throughput object storage for weights and logs, ingress/egress costs
- Monitoring and observability: token-level latency tracking, error rate dashboards, alert pipelines
- Orchestration: Kubernetes operators or equivalent for vLLM pods, autoscaling policy, load balancing
- On-call engineering: incident response coverage, especially for GPU failure modes that have no API equivalent
- Prompt and version governance: model version pinning, eval harnesses run on each weight update, rollback tooling
- Platform engineering FTE: the primary hidden cost — typically 0.5 FTE at minimum scale, 1.5 FTE for a multi-model self-hosted cluster
As the AuraByt self-hosted LLM TCO analysis notes, "The biggest hidden cost. Self-hosting an LLM in 2026 is much easier than it was in 2024, but it is not free of engineering work." A CTO who builds the budget around GPU spend alone will encounter the rest of these costs within 90 days of production launch.
Decision framework: which model bet fits which workload
Choose frontier APIs when capability is the constraint
- Choose frontier APIs (OpenAI, Claude) when: model quality is the primary product differentiator; complex reasoning, tool use reliability, or long-context accuracy is the bottleneck; volume is below 100M tokens/month; or the team lacks platform engineering capacity
- Routing signal: If swapping to a cheaper model in your eval suite drops task accuracy by more than 5 percentage points on your production test set, the capability premium is probably justified
The Artificial Analysis leaderboard provides the correct decision context here: it compares intelligence, pricing, speed, and latency side by side. Run your capability-bound tasks through that comparison before assuming the frontier premium is unavoidable.
Choose open weights when volume and control dominate
- Choose self-hosted or hosted open weights (Llama 3.1 70B, DeepSeek-V3, Kimi K3) when: monthly token volume exceeds 500M and traffic is predictable; data residency or audit requirements disqualify frontier APIs; cost predictability is a board-level concern; or the task (classification, extraction, summarisation) shows small quality gaps in your evals
vLLM's PagedAttention design makes open-weight deployment at scale practical: it reduces KV-cache waste and enables continuous batching. High GPU utilisation is achievable with sustained traffic, not bursty demand.
Use a three-layer portfolio instead of a binary bet
The binary framing — frontier API or open weights — is the wrong unit of analysis. A workload portfolio spans multiple task types with different quality thresholds, volume profiles, and residency requirements. A three-layer architecture serves this reality directly:
| Layer | Model options | Best for | Cost profile |
|---|---|---|---|
| Frontier hosted (Layer 1) | OpenAI, Claude | Reasoning, agentic tasks, capability-bound | $1–$5 in / $6–$30 out per 1M |
| Hosted open weights (Layer 2) | OpenRouter: Llama, Mistral Large, Kimi K3 | Mid-volume, mixed-sensitivity tasks, multi-model experimentation | $0.10–$0.80 in / $0.30–$2.50 out per 1M |
| Self-hosted fine-tuned (Layer 3) | Llama 3.1 70B / DeepSeek-V3 on H100, vLLM | High-volume, residency-constrained, domain-specific | $0.02–$0.08 in+out per 1M at utilisation |
Requesty and OpenRouter both illustrate that the market has converged on this pattern: "The AI Gateway for Production — 600+ models, real-time analytics, intelligent routing." Routing logic, eval governance, and model versioning become core platform assets in this architecture, not afterthoughts.
DecisionMatrix
| Workload archetype | Volume / risk signal | Sovereignty requirement | Recommended layer |
|---|---|---|---|
| Complex reasoning, agentic tool use, long-context coding | Under 100M tokens/month, high failure cost | No special requirement | Frontier hosted |
| Classification, extraction, summarisation | 100M–500M tokens/month, moderate failure cost | Optional | Hosted open weights |
| Predictable high-volume domain workflows | Above 500M tokens/month, stable traffic | Strong residency or audit need | Self-hosted open weights or bespoke fine-tune |
Risks, failure modes, and counterarguments
The primary risks to this portfolio strategy fall into four categories: vendor price shocks, model deprecation, capability jumps that invalidate routing decisions, and migration friction that proves larger than estimated.
Watch Out: Anthropic's announced pricing transition from $2/$10 to $3/$15 per 1M tokens after 31 August 2026 is a confirmed near-term price shock. OpenAI's tiered pricing creates exposure when a specific tier is deprecated or re-priced without the full tier range being maintained. Neither vendor's public pricing page constitutes a long-term pricing guarantee. Build budget scenarios that assume at least one vendor or model tier becomes unattractive during the 24-month horizon — because historically, that assumption has been correct.
Migration friction is consistently underestimated. Switching vendors requires rewriting prompts (system prompts, few-shot structures, and tool schemas are rarely vendor-neutral), revalidating eval suites on new outputs, reviewing safety filters for regression, and potentially re-running legal/DPA review. A model gateway reduces the architectural cost of switching but does not eliminate the prompt and eval work.
The hidden cost of waiting for perfect open models
The counterargument that open-weight models will soon eliminate the need for frontier APIs has been repeated at every major release since mid-2023, and it has always been partially correct and strategically misleading. Open weights have closed the gap substantially — DeepSeek-V3 and Llama 3.1 70B are credible for a large fraction of enterprise tasks — but frontier APIs continue to extend the quality frontier at the top.
Pro Tip: Sequence your model bets rather than freezing the roadmap. Start with frontier APIs to validate product assumptions and measure task-level quality requirements. When volume grows and quality thresholds are established, migrate cost-tolerant task types to hosted open weights. Invest in self-hosting only when the TCO analysis closes positively and residency requirements demand it. This sequencing preserves learning and avoids the opportunity cost of either waiting indefinitely for open models to catch up or committing GPU budget to self-hosting before traffic warrants it.
Procurement, compliance, and residency gotchas
Watch Out: Frontier API procurement involves legal review timelines that can run 30–90 days for enterprise BAAs and DPAs. Vendor data processing agreements may not satisfy GDPR Article 44 cross-border transfer requirements, FedRAMP requirements, or sector-specific mandates (HIPAA, SOC 2 Type II). Requesty and similar gateways advertise "EU data residency and enterprise governance" as a first-class feature — its prominence signals that residency is a standard procurement criterion, not an edge case. Regulated enterprises may find that certain frontier API offerings are disqualified from specific workloads regardless of benchmark performance, which alone can force an earlier move to self-hosted open weights than the cost model would otherwise suggest.
Which path should a CTO take in the next 12–24 months?
| Org profile | Recommended mix | Rationale |
|---|---|---|
| Startup / pre-scale | 90% frontier API, 10% hosted open weights via OpenRouter | Token volumes are low; engineering capacity is better spent on product; frontier APIs provide the capability headroom to iterate fast |
| Scale-up (growth-stage, >100M tokens/month) | 50% frontier API for capability-bound tasks, 40% hosted open weights (OpenRouter), 10% self-hosted for one high-volume workload | Volume justifies moving cost-sensitive tasks off frontier APIs; one self-hosting investment builds operational capability for future expansion |
| Regulated enterprise (financial, health, government) | 30% frontier API (residency-compliant deployments only), 30% hosted open weights (residency-verified provider), 40% self-hosted on sovereign infra | Residency and audit requirements drive the self-hosted layer regardless of cost optimality; gateway abstraction is mandatory |
Revisit the allocation quarterly. The Artificial Analysis leaderboard moves quickly, and the right routing decision at Q3 2026 may be wrong by Q1 2027 as open-weight models continue to close the quality gap on specific task types. Build the evaluation infrastructure to detect when a task-type migration becomes justified — don't rely on intuition about when open weights "are ready."
DecisionMatrix
| Org profile | Primary constraint | Best mix | Review cadence |
|---|---|---|---|
| Startup / pre-scale | Speed to product-market fit | Frontier API first, hosted open weights for volume tasks | Quarterly |
| Scale-up | Unit economics and routing flexibility | Frontier + hosted open weights + one self-hosted workload | Quarterly |
| Regulated enterprise | Residency, audit, procurement | Hosted open weights plus self-hosted sovereign infra | Monthly |
FAQ
Are open-weight models cheaper than API models? On a per-token basis at high utilisation, yes — a self-hosted DeepSeek-V3 or Llama 3.1 70B cluster can deliver tokens at $0.02–$0.08 per 1M versus $1–$30 per 1M for frontier APIs. But total cost of ownership for self-hosted LLMs includes GPU infra, monitoring, orchestration, on-call, and 0.5–1.5 FTE platform engineering overhead. At low to medium volume, frontier APIs are frequently cheaper when all costs are counted.
When is self-hosting an LLM worth it? Self-hosting is financially justified when monthly token volume exceeds roughly 500M tokens with predictable, sustained traffic — not bursty load — and GPU utilisation can be maintained above 60–70% through continuous batching. It is also justified regardless of volume when data sovereignty, residency mandates, or audit requirements disqualify frontier API providers.
What are the risks of relying on a single frontier model vendor? Price changes, model deprecation, rate-limit changes, and capability jumps that force prompt rewrites. Single-vendor dependence also concentrates procurement risk: if one provider's DPA is rejected in legal review, you have no fallback.
How do open weights compare to frontier models for enterprise use? Open weights are competitive on classification, extraction, summarisation, and retrieval tasks — particularly with fine-tuning on domain data. They trail frontier models on complex multi-step reasoning, agentic tool orchestration, and novel instruction-following at the top of the capability distribution. The gap is shrinking but remains real in 2026.
Pro Tip: When running a procurement comparison between OpenAI pricing and Claude API, compare five dimensions simultaneously: input and output token price, rate limits at your projected peak traffic, context window and throughput at that window, data residency and DPA availability, and migration friction (tool schema compatibility, structured output behaviour, eval regression risk). Token price alone produces the wrong procurement decision.
Is it better to use OpenAI or Claude for production workloads? Neither dominates across all workloads. Claude is competitive on output-token price through August 2026. OpenAI offers multiple tiers that let you map spend to workload criticality. The correct answer is workload-specific: run your production eval suite against both, measure quality and latency on your actual task distribution, and route accordingly — preferably through a gateway that lets you shift allocation without redeployment.
Sources and references
- OpenAI API Pricing — Official per-token pricing for GPT-5.6 tiers; captured August 2026
- Anthropic Claude Pricing — Official pricing page documenting the introductory $2/$10 and post-August 2026 standard $3/$15 rates
- Artificial Analysis Model Leaderboard — Comparative intelligence, pricing, speed, and latency rankings across frontier and open-weight models; snapshot August 2026
- vLLM Blog — PagedAttention — Technical documentation of PagedAttention KV-cache management for high-throughput open-weight inference
- OpenRouter Models Page — Multi-provider API aggregator covering 400+ models with unified pricing and benchmarks
- Requesty AI Gateway — Production model gateway with intelligent routing, EU data residency, and enterprise governance features
- AuraByt — Self-Hosted LLM TCO 2026 — 2026 analysis of total cost of ownership for self-hosted large language models including hidden engineering overhead
- Agaro — Foundation Model Portfolio Strategy — Source strategy piece recommending a three-layer portfolio of frontier-hosted, self-hosted, and fine-tuned models for enterprise AI planning
Keywords: OpenAI pricing, Claude API, self-hosted LLMs, Artificial Analysis leaderboard, GPT-5.6, Claude 3.5 Sonnet, Kimi K3, Llama 3.1 70B, DeepSeek-V3, Mistral Large, H100, vLLM, model gateway, API rate limits, vendor lock-in
