Independence
We sell tests. We publish what we find.
CrashMire tests AI agents and publishes what it finds: the method, the figures, the conversations. We also sell crash tests, and we say so everywhere. Here is what is paid, what stays free, and the rules we hold ourselves to.
What we sell
Crash tests of AI agents for companies, follow-up over time and quoted offers for large accounts. It is our trade: we say so on our pages, in our talks and at the top of our mandatory-testing proposal. We take no commission from any model provider. A company, or a model's publisher, may pay for a model the barometer does not show yet to be measured: the same price for all, and the price never depends on the result.
What stays free and open
- The public barometer: figures and conversations under CC BY 4.0
- The method, public and versioned
- The Léa demo, open to all
- Tool Guard, an open-source Symfony bundle (MIT licence) that blocks out-of-rule actions in code · Tool Guard and CrashMire together
- The mandatory-testing proposal: it asks for a test by the body the company chooses, not by us
How we publish
- Every result is dated and tied to the exact identifier of the model tested: a model changes, a figure goes stale.
- From the December 2026 edition, each provider receives its models' results before the barometer is published, so it can answer.
- We also publish what does not suit us: the models that hold, and the simple fixes that are enough without a crash test.
- A reported and verified mistake is corrected on the page, with its date.
- A model measured on demand carries its mark in the barometer: "measured at a client's request" (never the client's name), or "requested by the publisher". A result requested by a publisher is published whatever it is.
What we will never claim
- That an agent is "certified", "compliant" or "guaranteed": a test shows what was observed, not the absence of risk.
- A figure we did not measure.
- A client's name without their written consent.
- That you must go through us: a test done seriously by someone else is worth ours.
We are not the only ones measuring
INESIA, the French national institute for the evaluation and security of artificial intelligence, research labs and other testing companies also publish on the safety of models and agents. That is good news: the more public measurements there are, the more companies improve. Our comparisons say honestly where each is most useful. Read the comparisons