AI Made

Get the Newsletter

Open Source AI vs Closed AI in 2026: The Honest Trade-offs

Llama 4 Maverick beats GPT-4o on HumanEval by a hair, costs 10x less per output token, and ships under a license that flips to a private contract the moment your product passes 700 million monthly active users. The “open source AI” label in 2026 is doing a lot of work, and most of it is wrong. Here is what the open-vs-closed label actually controls, what it does not, and how to decide which side to bet on for a workload you intend to ship.

The honest comparison is not “open is better” or “closed is better.” It is a four-axis trade-off — capability, privacy, cost, and operational overhead — and each axis has a different answer depending on what you are building. Three things changed in 2026 that make the conversation different from 2024: open-weight models closed the coding gap to single-digit percentages, the EU AI Act created a legal exemption for open-source GPAI models, and inference economics flipped at the high-volume tier. This post walks through each axis with primary sources, then gives you a decision framework you can actually use.

What “Open Source AI” Actually Means in 2026

The first thing to settle is the vocabulary, because marketing copy is doing the work that technical definitions should do. The Open Source Initiative published the canonical distinction in October 2024 and updated it through 2026: open-source AI means sharing the training code, the training dataset where legally possible, the data composition, and the model weights. Open weights means sharing only the final trained model — the file of numbers you download and run. You get a working model in both cases, but you cannot audit, retrain, or fully customize the second one.

The honest comparison is a deployment-framing question. The chart below shows the open-source vs closed-source trade-off in deployment terms: smaller, open-weight models running on local hardware versus larger, closed-source models accessed via cloud API. Both paths work. The right one depends on what you actually need.

Editorial bar chart comparing on-device and cloud AI model classes on accuracy and latency — visual metaphor for open-source local deployment versus closed-source cloud API

Three license families cover most of what ships as “open” today:

  • Apache 2.0 and MIT — truly permissive. You can fine-tune, redistribute, and commercialize without restriction. Mistral’s Mixtral 8x7B (released Dec 2023 under Apache 2.0) set the precedent. Qwen3-235B-A22B from Alibaba (Fireworks deployment page) is the current 2026 benchmark for “open-source weight, permissive license, frontier capability.”
  • Custom community licenses — Meta’s Llama 4 Community License Agreement is the textbook case. Weights are free to download. Redistribution requires a “Built with Llama” attribution and inclusion of the license text. Above 700 million monthly active users in any product you ship, you must request a separate license from Meta “in its sole discretion.” You are not authorized to use the model commercially until Meta grants that license. This is open weights with strings, not open source.
  • Bespoke / “modified” licenses — Moonshot’s Kimi K3 shipped in July 2026 under a custom license that requires a separate agreement for any Model-as-a-Service business with more than $20 million in annual revenue. Hugging Face categorizes the license as “other,” not as an SPDX-recognized identifier. Community discussion on Reddit quickly mislabeled it as “Modified Apache” — it is not.

The pattern across 2024–2026: open-weight releases are growing more capable, but the licenses are growing more restrictive. Apache 2.0 / MIT is now the minority, not the default. If you build a product on top of Llama 4 today, you should plan for the 700M-MAU clause as a real business risk, not a hypothetical. If you build on Qwen3 or DeepSeek-R1 (MIT), you do not have that risk.

Where Open-Source Models Are Genuinely at Parity

The capability gap closed faster than almost anyone predicted. Epoch AI’s Capabilities Index for the Jan–May 2026 window puts the open-weight lag at an average of four months and 8 ECI points (90% CI 7–11) — actually slightly wider than the three-month lag Epoch measured for the preceding two years. So the gap did not keep shrinking in 2026. It leveled off. But “four months behind the frontier” is a categorically different statement from “two years behind,” which is where the conversation lived in 2024.

On specific tasks the gap is functionally zero. HumanEval, the Python-coding benchmark, has both ends sitting in the saturation regime — DeepSeek V4.1 Pro at roughly 97.8% versus Claude Mythos 5 at roughly 98.8% (one point). On instruction following (IFEval), open-source models now score above 85% versus GPT-4o’s 88%, a difference that disappears in real applications. On factual knowledge (MMLU), the gap is approximately 8–10% — real, but not catastrophic.

Llama 4 Maverick, released April 2025, is a 17-billion-active-parameter MoE with 128 experts that fits on a single H100 DGX host at FP8 quantization. On HumanEval, the 405B variant actually outperformed GPT-4o by a small margin — the first time an open-weight model beat GPT-4o on a standard coding benchmark. For teams that need to write Python functions from docstrings, the open-vs-closed question is now a cost question, not a capability question.

The most striking recent example: Zhipu AI’s GLM-5.1 (April 2026) is a 744B-parameter coding model under MIT license that hit SWE-Bench Pro 58.4 — competitive with what Anthropic’s Claude Opus 4.7 was doing twelve months earlier. For teams running high-volume coding workloads under data-sovereignty constraints, that combination of capability and licence is the threshold where self-hosting stops being a curiosity and starts being the default.

Where Closed Models Still Lead

Three places where the gap is still meaningful and the trajectory is slower than the headlines suggest.

Agentic tasks. On WebArena (browser-use), OSWorld (computer-use), and TerminalBench (shell-use), the closed-model advantage holds at roughly 12–14 points — GPT-5.6 Pro at 62% on WebArena versus DeepSeek V4.1 Pro at 48%. The likely cause is that agentic capability depends as much on tool-use scaffolding and post-training as on the weights, and closed labs can iterate on scaffolding faster than they can release weights. When you download a model file, you get the weights. You do not get the proprietary RLHF recipe or the tool-use tuning corpus.

Private benchmarks. Lumiere Research’s analysis of Epoch AI’s data and CAISI’s independent evaluation makes the leaderboard-vs-reality gap explicit. CAISI’s private benchmarks — ARC-AGI-2 semi-private, the held-out software-engineering evaluation PortBench, and CTF-Archive-Diamond — put DeepSeek V4 Pro’s capabilities about eight months behind the frontier, twice the leaderboard-derived estimate. Epoch is careful about this: they note that open-weight models tend to perform worse on private benchmarks than on public ones, plausibly because they hill-climb public benchmarks more aggressively during post-training. Nobody has isolated training-set contamination as the mechanism. The consistent gap between public and private results is what matters for high-stakes deployments.

Long-horizon knowledge work. On AA-Briefcase, Artificial Analysis’s proprietary long-horizon evaluation (thousands of complex input files, scored on correctness and analytical quality), Kimi K3 placed second overall behind Claude Fable 5. The score was close — rubric scoring and analytical quality came within a few points — but the closed model won. For agentic sessions that have to produce a deliverable at the end of a 30-minute run, the closed-model advantage persists.

Privacy: What “On-Premises” Buys You and What It Does Not

Privacy is the most overclaimed benefit on both sides, so it is worth separating what you actually get.

Open-source gives you architectural control. When you run Llama 4 or Qwen3 in your own VPC, on-prem, or fully air-gapped, no third party touches the inference path. There is no provider to subpoena, no provider to change terms, no provider whose outage takes your product down. For regulated industries — healthcare (HIPAA), finance (PCI-DSS, SOC 2), defence, public sector — this is the entire reason to consider open-weight at all. The Cohere Sovereignty framing and Prem AI’s list of EU-sovereign LLM APIs are explicit about this: the value is not cost, it is jurisdiction.

Closed-source has matured on data retention. A 2026 study on zero-data-retention in enterprise AI assistants mapped the practical landscape: Salesforce AgentForce relies on contractual ZDR with third-party LLM providers; Microsoft Azure OpenAI Service hosts OpenAI models directly in-tenant (architectural isolation, not contractual); Anthropic offers ZDR for enterprise but requires explicit configuration; Google Gemini retains 24 hours by default. DeepSeek retains 30 days and offers no ZDR option — the paper explicitly calls it “unsuitable for regulated industries.”

The EU AI Act created a legal exemption. Under Article 53(2) of the EU AI Act (see our full AI Regulation in 2026 guide), providers of open-source GPAI models are exempt from the technical-documentation and downstream-info obligations IF the model is released under a free-and-open-source license, weights + architecture + usage info are publicly available, and the model is NOT classified as systemic-risk. The exemption does NOT cover the training-data-summary obligation (Article 53(1)(d)) or the copyright-policy obligation (53(1)(c)). The European Commission’s Article 50 implementation guidelines, published July 2026, narrow the exemption further: monetisation (charging for technical support, monetising a platform around the model, or using personal data for non-improvement purposes) LOSES the open-source exemption. Practically, this means an open-source GPAI released under Apache 2.0 with monetised hosting still has to comply with Article 53(1)(c) and (d). The licence does not exempt the operator, only the original releaser.

The risk profiles differ. Open-source carries operational risk: you own the security perimeter, the patching cadence, the access controls. Closed-source carries contractual risk: the provider’s policy can change on notice, and an outage takes your product down. For most regulated use cases, the practical answer is “both” — keep customer-facing sensitive data on-prem with open weights, route general traffic to a managed closed API with explicit ZDR configuration.

The Real Cost Numbers: API vs Open-Weight Hosted vs Self-Hosted

The headline cost comparison is misleading. Per-token, open-weight hosted looks 10x cheaper than GPT-4o. The total cost of ownership over three years tells a different story.

Per-token rates as of mid-2026 (from the Simplifai pricing comparison), input / output per million tokens:

ModelInputOutputContext
GPT-4o$2.50$10.00128K
GPT-4.1 nano$0.10$0.401M
Claude Opus 4$15.00$75.00200K
Claude Haiku 3.5$0.80$4.00200K
Gemini 2.5 Flash$0.15$0.601M
Llama 4 Maverick (Together)$0.27$0.851M
Llama 4 Scout (Together)$0.18$0.5910M
DeepSeek V3 (DeepSeek API)$0.27$1.10128K
DeepSeek V4 Pro (DeepSeek API / Venice)$2.40$4.801M

Output tokens cost 3-5x more than input across every provider. This is an industry-wide pattern, not a one-vendor quirk. If your workload is generation-heavy (summarization, long-form content, code generation), the output-token multiplier is what dominates the math. Llama 4 Maverick hosted on Together at $0.85/M output is roughly 12x cheaper than GPT-4o and 88x cheaper than Claude Opus 4 on the same axis.

DeepSeek’s own API pricing for V4 Pro — $2.40/M input, $4.80/M output for a 1M-context model hitting MMLU 87.8% — is the latest reference point for “comparable quality at 8–20x lower cost than GPT-5.5 / Opus 4.7.” For generation-heavy production workloads, the open-weight economics are no longer debatable.

Three-year TCO tells a different story at the heavy tier. Ginger Labs’ 36-month TCO model at 50M tokens/day (3:1 input-output ratio):

Tier (50M tok/day)Year 136-MonthNotes
OpenAI GPT-4.1$126,000$378,000API only, no infra
Anthropic Sonnet 4$180,000$540,000API only
Open-weight hosted (Together/Fireworks)$36,000$108,000API, no infra
Local enterprise (4× H200)$308,347$391,707Hardware amortises

The crossover happens around 5M tokens/day. At 5M tok/day, 36-month TCO is roughly $32,870 for local consumer hardware versus $37,800 for OpenAI API. At 50M tok/day, the open-weight hosted option is the cheapest by a wide margin ($108k vs $378k for OpenAI, $540k for Anthropic). Self-hosting at the heavy tier closes the gap with OpenAI by month 36 because hardware depreciates to zero, but you carry the operational risk and the hardware refresh cycle.

The 2026 shift: break-even points are roughly 40% lower than 2024. Better open-weight models (Llama 4, Qwen3, DeepSeek-V3), cheaper hardware, and more efficient serving stacks all moved the line. Below 1M tokens/day, API is still simpler and cheaper. Above 10M tokens/day, open-weight hosted dominates. Above 50M tokens/day with a regulatory driver, self-hosting earns its operational cost.

The Self-Hosting Stack in 2026: Ollama, vLLM, llama.cpp

If you decide to self-host, three runtimes cover almost every case. They each optimize a different point on the same trade-off curve, and you often end up using more than one.

vLLM is the production-grade shared-API choice. Built at UC Berkeley around PagedAttention (the same memory-management trick that lets operating systems page virtual memory, applied to the key-value cache) and continuous batching, vLLM is the only engine that holds latency stable under double-digit concurrency. The Red Hat benchmark on Llama 3.1 8B at FP16 shows both engines comparable at a single concurrent request, then vLLM pulling away as concurrency rises — generating roughly 44 times more tokens per second than llama.cpp at 64 simultaneous users, the gap driven by vLLM’s continuous batching versus llama.cpp’s sequential queue. Time-to-first-token diverges the same way: llama.cpp’s queuing model leaves later arrivals waiting in line, while vLLM interleaves token generation across the batch. vLLM requires an NVIDIA CUDA GPU — minimum 8GB VRAM for a 7B AWQ model, 24GB+ recommended. There is no CPU fallback.

llama.cpp is the hardware-flexible choice. It is a pure C++ implementation with essentially no external dependencies, written by Georgi Gerganov in 2023 to run LLaMA on a MacBook CPU. It created the GGUF format that every other consumer-grade runtime now uses, and it runs anywhere: CPU, Apple Silicon via Metal, NVIDIA via CUDA, AMD via ROCm, Vulkan-capable hardware, Raspberry Pi. Quantization is its other strength — Q4_K_M hits roughly 75% size reduction from FP16 with under 2% quality degradation on standard benchmarks, and the model fits in 4-5GB VRAM. The limitation is concurrency: llama.cpp processes requests in a queue. One request at a time. Fine for personal use, painful for a shared API.

Ollama is the developer-experience wrapper. Built on top of llama.cpp, Ollama adds model management (pull, run, list, delete), a simplified REST API, and Modelfile-based customization. The whole local setup is three commands: ollama pull llama3 , ollama run llama3 , done. Ollama does not implement PagedAttention or continuous batching, so P95 latency exceeds 60 seconds once concurrency passes 10 users. Treat it as the fastest path to a working system and the development-side twin of a vLLM production tier.

The practical deployment pattern: prototype with Ollama locally. Once the prompt and model are settled, move to vLLM for the shared staging server. If the feature needs to work offline or on edge hardware, port the llama.cpp version. All three speak OpenAI-compatible HTTP, so application code rarely changes between tiers.

Self-hosted GPU server racks with cyan LED cable management — visual metaphor for the open-weight inference infrastructure path (vLLM, llama.cpp, Ollama)

The above is the open-weight deployment path in practice: GPUs you own, cable management you manage, and inference budgets you control. The capital cost is real (an 8×H100 node runs roughly $200K–$300K depending on the generation), but it amortises to electricity and ops labor at month 36 — which is when the TCO math starts favoring self-hosting for medium-volume workloads.

One last deployment gotcha worth knowing: Moonshot’s Kimi K3 weighs 1.56 TB across 96 safetensors shards. Moonshot’s own technical write-up recommends a supernode of 64 or more accelerators for efficient inference. Secondary reports put the floor at 8 B300 GPUs, 16 H200s, or 32 × 80 GB H100s. The “you can run it on a laptop” story for open-source AI is true for 7B–30B dense models and MoE models with under 30B active parameters. Anything beyond that is datacenter-class hardware.

Decision Framework: Which Side Should You Pick?

Six concrete scenarios, mapped to the answer that fits:

  • You’re building a customer-facing chatbot on sensitive data (healthcare, finance, legal): Self-host an open-weight model with strong Apache 2.0 or MIT license (Qwen3, DeepSeek-R1, Mistral) in your own VPC or on-prem. Use vLLM for the inference server. Skip the cloud-API path entirely; the audit trail is the product.
  • You’re shipping a coding assistant for an internal team: Open-weight hosted on Together or Fireworks at $0.27/M input. The cost gap versus GPT-4o is 10x, the capability gap is functionally zero on HumanEval, and the operational overhead is zero. If your team is more than 20 engineers, self-host on vLLM behind nginx with API key auth.
  • You’re building an agentic application with tool-use, browser-use, or computer-use: Closed-source for now. The 12-14 point gap on WebArena, OSWorld, and TerminalBench is the difference between a demo and a production system. Re-evaluate every six months — Kimi K3 placed second on AA-Briefcase, the long-horizon knowledge-work evaluation, so the agentic gap is narrowing.
  • You’re processing >10M tokens/day on a budget: Open-weight hosted (Together/Fireworks/Groq). At that volume the per-token savings pay for the migration effort in weeks. Use the savings to invest in evaluation infrastructure — private benchmarks, A/B testing, and human review — so you know whether your cheaper model is actually doing the job.
  • You’re shipping to the EU market and care about the open-source GPAI exemption: Pick a model under a truly permissive license (Apache 2.0 or MIT), publish the training-data summary and copyright policy yourself, and document the operational architecture. Article 53(2) exemption requires the model to be released under a free-and-open-source license, weights + architecture publicly available, and NOT a systemic-risk model. Monetised hosting does not lose you the exemption if you meet those three.
  • You’re shipping an application that needs the absolute best reasoning capability: Closed frontier (GPT-5.6 Pro, Claude Opus 5, Gemini 3.2 Pro) for the hard queries, open-weight for the long tail. The multi-model router pattern — front the closed model with a cheap open-weight classifier that decides which queries deserve the expensive inference — saves 90%+ versus routing everything to the frontier model.

What to Do This Week

  • Audit your current AI spend against the open-weight alternatives. Pull last month’s token usage by provider. Calculate what Llama 4 Maverick on Together or DeepSeek V3 on DeepSeek’s API would have cost at the same volume. The output-token multiplier is where the savings live.
  • Run the OSI definition on every model in your stack. Open-source (training code + data + weights) versus open-weight (weights only) versus closed (API only). If you are relying on the “open source” label for procurement, compliance, or marketing, the distinction matters.
  • Test one workload on a self-hosted open-weight model. Pick a non-customer-facing internal task (code review, log analysis, document summarization). Run ollama run llama3.2 on a developer laptop, then vllm serve on a single H100. Measure latency, throughput, and quality against your current API provider. The data tells you whether self-hosting is worth the operational cost.

Frequently Asked Questions

Is open source AI actually free to use?

The weights are free to download. Operating them is not. A 235B MoE like Qwen3 needs an H100 cluster; Kimi K3 requires 64 accelerators minimum for efficient inference. Apache 2.0 models (Qwen, Mistral) let you fine-tune and commercialize the output without restriction. Meta’s Llama 4 license flips to a private contract above 700 million monthly active users.

Can I fine-tune an open-source model on my own data?

Yes, on Apache 2.0 and MIT models — those allow derivative works without restriction. The practical constraint is GPU time: QLoRA on a 70B model fits on a single A100, while full fine-tuning on a 235B model needs multi-node H100 clusters. Llama 4 has a custom license that requires “Built with Llama” attribution and triggers a separate commercial agreement above the MAU threshold.

Which is safer for privacy — open source or closed?

Open-source gives you more control: you can run the weights in your own VPC, on-prem, or fully air-gapped, with no third-party data transmission. Closed-source offers managed zero-data-retention tiers — Salesforce AgentForce and Microsoft Azure OpenAI both architecturally isolate customer data. The risk profiles differ: open-source carries operational risk (you own the security perimeter); closed-source carries contractual risk (the provider’s policy can change). For healthcare and finance, the practical answer is “both,” with different workloads on each.

Is OpenAI’s GPT or Anthropic’s Claude open source?

No. Both are closed-source — weights, training data, and training code are proprietary. OpenAI and Anthropic offer API access and managed deployment but do not release model artifacts. The leading closed models in 2026 are GPT-5.6, Claude Opus 5 / Fable 5, and Gemini 3.2 Pro.

How do I start self-hosting an open-source model?

Three-step path: (1) install Ollama and run ollama run llama3.2 to validate the workflow on a laptop; (2) move to vLLM on a dedicated GPU server when concurrency exceeds 5 users; (3) graduate to llama.cpp only if you need to run on CPU, Apple Silicon, or air-gapped hardware. Each tier uses the same OpenAI-compatible API surface, so application code rarely changes between them.