TL;DR
- “Open weights” describes access to weight files. It does not, by itself, establish full disclosure of how a model was built or tested.
- In the 2025 Foundation Model Transparency Index, the five developers with open-weight flagships had widely different disclosure scores. Those scores measure what firms disclose. They do not measure how well a model works.
- A separate Radboud study of 45 models billed as open found openness uneven and often overclaimed. It treats openness as several parts, not one switch. Ask what is open in the exact release you plan to use.
- Keep a release record. Name what you can get, what is missing and what was checked. A download alone does not prove that the whole model is open.
A Download Does Not Answer Every Buyer Question
A buyer may want weight files for local use. Another may need the training recipe to repeat a result. A third may need to check test methods. These needs overlap, but they are not the same. A broad “fully transparent” label can hide the missing answer.
For generative engine optimization (GEO), the source page should state which artifacts it offers for which release. An AI answer can then preserve that scope. The goal is a claim a buyer can check. A vague label may mean one thing at the top of the page and another at the end.
Weights, training data and code are separate parts of a release. A paper can show a model’s design yet leave out the full training recipe. A test report can give results yet leave a reader unable to rerun the test. A release can be useful even when some of these parts are not public.
Open-Weight Flagships Had Different Disclosure Coverage
The 2025 Foundation Model Transparency Index assessed 13 developers using 100 indicators. The authors used company reports and public sources. Seven firms sent reports. For the other six, the team used human search and an automated search agent to gather public information. All 13 firms could respond to initial scores before the scores were finalized. Two researchers scored each item on their own, then resolved their differences.
The report classed five of the flagship models as open weight. Their developers’ overall scores were AI21 Labs 66, Alibaba 26, DeepSeek 32, IBM 95 and Meta 31, out of 100. Figure 1 includes all five in alphabetical order. The scores belong to that 2025 study. They do not rank every model those firms offer today.
Each item gets zero or one point. A score of 66 thus meets 66% of the index’s 100 indicators. This is a score for what a firm discloses. It is not a 66% chance that a model is safe, right or a good fit.
2025 FMTI, Table 1, Figure 7 and Figure 9. All five developers whose assessed flagship was designated open weight. Scores cover wider developer disclosures as defined by the index. They do not certify a license, model quality or completeness of a training-data release. Original index, scores and method.
A Separate Study Treated Openness as Several Parts
Radboud University’s July 2024 summary of its own research describes Liesenfeld and Dingemanse’s review of 45 text and text-to-image models presented as open. The team found openness uneven and often overclaimed across models billed as open. It treats openness as made of several elements and as a matter of degree. The summary names code, training data, fine-tuning data and model design as areas to inspect.
The two studies have different scopes. The Radboud team looked at model openness. FMTI scored what firms disclose across a wider set of fields. Its total does not replace a check of the exact files a buyer plans to use.
| Independent evidence | Actual finding and scope | What the result does not establish |
|---|---|---|
| 2025 FMTI original report | 13 developers, 100 binary indicators; five open-weight flagships span 26–95 indicators satisfied | A quality ranking, security test or proof that each training artifact is public. |
| Radboud research summary, July 2024 | 45 models billed as open; openness uneven and often overclaimed; a composite view covers code, data and architecture | A current audit of all releases, or a common score that can be pooled with FMTI. |
Figure 2. Evidence comparison. Different author teams, units and assessment methods. One measures developer disclosure coverage; the other studies model openness as a composite, graded property. Neither turns a weight download into proof of full disclosure.
Do not infer that open weights have no value. They can offer a concrete artifact to inspect or use, subject to the release terms. Nor do the studies show that low disclosure proves a model performs poorly. What is public and how well it works need separate proof.
The findings do not settle the license or legal duties for a current release. Here, they help narrow a broad claim. They do not apply a legal test to a named firm.
A Completed Release Record for a Transparency Claim
This example is fictional. A page calls model K1 revision 2 “fully transparent.” The reviewer holds a release packet with downloadable weights, named inference code and a model card. No real supplier package was audited, no model was run and no license conclusion is implied.
The table turns the broad label into claims that can be checked. “Not public in this record” means the held packet lacks a public source for that field. The item may still exist. The gap alone does not show bad conduct.
| Fictional release record | Finding for K1 revision 2 | Publishable scope or next check |
|---|---|---|
| Weight files: download and version recorded. | The specified weight files are available in the held packet. | Name that release and the files; do not infer access to its training data. |
| Named inference code and model card: public links recorded. | Those artifacts are disclosed. A full training recipe is not provided. | Name the inference code; avoid saying all development code is public. |
| Training-data provenance: no public artifact in the packet. | The record does not support full training-data disclosure. | Mark that field as not public in this record; ask the technical owner for any source. |
| Evaluation report: methods and results provided; no independent rerun held. | Reported tests are available. Reproduction has not been verified. | Attribute the results to the report; do not claim an independently reproduced model. |
| Draft claim: “K1 revision 2 is fully transparent.” | The wording exceeds the disclosed artifacts. | Replace the broad claim with the statement below; technical owner and editor check every named artifact. |
The revised statement is: “K1 revision 2 provides weight files, the named inference code and a model card. Full training-data provenance and a full training recipe are not public in the record reviewed. Evaluation results are reported; independent reproduction has not been verified.”
The technical owner checks the links, files and version. The editor checks that the text names only the items held. A later release needs a new record. The finding does not cover every model from the same firm.
Keep this separate from whether a provider trains on or retains customer inputs. A public release record shows what a firm shares. Its rules for customer inputs concern what happens to a buyer’s own data. One claim does not settle the other.
Common Questions
Can we call a model open source because its weights are downloadable?
A download alone is not enough to establish that claim. Check the exact release terms and the artifacts available. This article does not classify licenses. Safer factual wording names the weights and code that are actually provided.
Does a high FMTI score prove better model performance?
No. The score measures what a firm discloses under the index’s indicators. It does not measure how often a model gets a task right or how safe it is. Check those with tests suited to the buyer’s task and model version.
Does missing training data mean the model cannot be used?
That does not follow from the gap alone. A buyer may need to use a model, inspect it or rebuild it. Each need calls for different files. Name the missing item, then check which need it affects.
Should an AI answer repeat the developer’s openness label?
It should keep the label’s source and scope, then name the files and facts that support it. A source calling weights open does not support a new claim that all code, training data and tests are public.
Make the Release Claim Easy to Check
Bring the approved release page, artifact links and saved AI answers to a GEO review. SemShift can help find where broad labels outrun the source record. The practical goal is wording that preserves what is known and what remains open, without a promise of AI citation gains.