“Open Source” AI Usually Isn’t. Here’s the Three-Part Test.

A museum display case: an assembled wristwatch at the centre, surrounded by every one of its components laid out on labelled blocks.

Download Llama and you will see it described as open source in roughly half the places you look, including places that should know better. It is not, and the difference is not pedantry. It decides whether you can use the model in a product, what you have to put on your website, and whether you can ever find out what the thing was trained on.

The useful part is that there is a published test with three parts, and you can run it yourself in about five minutes per model. Here it is, along with what the licences actually say.

This article is based on the licence texts and documentation cited below, read on 4 October 2026. No models were benchmarked or tested for it. Licences change, so the dates matter — each claim below is dated.

The word does two different jobs

“Open” in AI gets used for two quite different things.

Open weights means you can download the trained model and run it. You can fine-tune it, host it on your own hardware, and use it offline. What you cannot do is find out how it was built, because the training data and the training code were not published.

Open source, in the sense the term has meant in software for decades, means you also get the recipe — enough to rebuild the thing yourself and audit what went into it.

Almost every model marketed as “open AI” is the first kind. That is still genuinely useful. It is just not the same promise, and the gap is where people get caught.

The three-part test

In 2024 the Open Source Initiative — the body that has maintained the definition of open source software since 1998 — published version 1.0 of an Open Source AI Definition. It asks for three things, and the wording is specific.

Data information. “Sufficiently detailed information about the data used to train the system so that a skilled person can build a substantially equivalent system.” Note what this does not demand: the definition does not require shipping the dataset itself, which would be impossible for a lot of licensed material. It requires enough detail to reproduce it.

Code. “The complete source code used to train and run the system.” Training scripts and data processing, not just the inference code that runs the finished model.

Parameters. “The model parameters, such as weights or other configuration settings,” including intermediate checkpoints.

All three have to come with four freedoms: to use the system for any purpose, study how it works, modify it, and share it.

Run that test against the models you have heard called open source and most of them fail on the first two parts. They ship the parameters. That is the whole of it.

What the Llama licence actually says

A printed contract headed Terms and Conditions, a fountain pen resting across the signature block.
Photo: Blogtrepreneur / Wikimedia Commons, CC BY 2.0. Header image: Kozuch / Wikimedia Commons, CC BY-SA 3.0.

Llama is the usual example, so it is worth quoting rather than characterising. The Llama 4 Community License Agreement, effective 5 April 2025, is not an OSI-recognised licence, and two clauses are the reason.

The first is a size cap on who may use it freely:

If, on the Llama 4 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta

Seven hundred million monthly users is not a threshold most readers of this site will trouble. That is rather the point of it — the clause is aimed squarely at a handful of rival platforms. But a licence that grants rights to everyone except a named competitor class is doing something an open source licence cannot do. The first of the four freedoms is use for any purpose, by anyone.

The second clause costs you something immediately:

prominently display “Built with Llama” on a related website, user interface, blogpost, about page, or product documentation. If you use the Llama Materials or any outputs or results of the Llama Materials to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, you shall also include “Llama” at the beginning of any such AI model name.

So: attribution on your product surface, and if you fine-tune Llama into something you ship, the thing you ship has to be called Llama-something. If you were planning to fine-tune a model and give it your own product name, read that sentence twice.

Neither clause makes Llama a bad deal. Both make it a licence with conditions, which is the opposite of what people assume when they hear open source.

What open weights still get you

It is worth being clear that the open-weight models are not a consolation prize. With the weights alone you can:

  • run the model on your own hardware, with no per-token bill and no data leaving your network;
  • keep using a specific version after the vendor has moved on, which is not a small thing if you have built around one model’s behaviour;
  • fine-tune it on your own examples;
  • inspect and modify the inference code and the way the model is served.

For most practical purposes that covers it. If your reason for wanting an open model is cost control or data residency rather than auditability, open weights do the job, and the cost arithmetic usually favours them at volume.

What you cannot do is answer the question “what is in this model”. If that question matters to you — for regulatory reasons, for research, or because you want to know whether your own scraped posts are in there — the weights alone will not tell you.

The models that publish all three parts

They exist, and they are mostly from research institutions rather than product companies.

The clearest example is Olmo from the Allen Institute for AI, a non-profit research institute. Ai2 states that “the entire model flow – not just its endpoint – should be accessible and customizable”, and says that “unlike current open LLMs, which limit access to their training data, architectures, or evaluation methodologies, Olmo stands out by providing full access.”

That is a vendor’s claim about itself, so check it against someone else. The European Open Source AI Index, run with support from the Centre for Language Studies and the Dutch Research Council, assesses models on availability, documentation and licensing. Its Olmo entry records the pretraining data released and documented, the fine-tuning data published, base and end-model weights on Hugging Face, training code across two repositories, a datasheet, a preprint and a peer-reviewed paper, under “Apache 2.0, an OSI-approved license.” Olmo 3 was released in November 2025 at 7B and 32B sizes.

The same index’s Llama 4 entry, last updated 13 March 2026, reads differently: the licence is flagged as “not an OSI recognised open license”; the training-code repository “only offers code for inference pipeline”; there is no datasheet and no peer-reviewed paper; and the base model data is described in Meta’s own words as “a mix of publicly available, licensed data and information from Meta’s products and services,” including “publicly shared posts from Instagram and Facebook.”

One of those is an open source model. The other is a free download with conditions.

How to check a model in five minutes

You do not need to read a full licence to place a model. Four checks, in order:

  1. Find the licence name. If it is Apache-2.0 or MIT, it is a standard open source licence and the parameters are genuinely open. If it has the model’s name in it — “Llama 4 Community License”, “Gemma Terms of Use” — it is a custom licence, and custom licences exist to add conditions.
  2. Search that licence for a number. Monthly active users, revenue, employee counts. A threshold means the grant is conditional on your size.
  3. Look for a naming or attribution clause. Search for “display”, “attribution”, “name”. This is the one that most often surprises people shipping a product.
  4. Open the training-code repository. If it only contains inference code — how to run the model rather than how it was trained — the second part of the test has failed, whatever the README says.

If you want the work done for you, the European index above covers around 150 models and shows its reasoning per dimension, which is more than most vendor pages do.

What to remember

Open weights are a licence to run. Open source is a licence to inspect and rebuild. Most of what gets called open source AI is the first, and the models that are genuinely the second tend to come from non-profit labs rather than the companies with the biggest models.

Before you build on one, read the licence for a number and a naming clause. Those two things cost more projects a rewrite than anything in the model card.