It says "verified." Verified against what?

TL;DR

• On October 22, Anthropic published research on formally verifying code from 123 libraries. The same page says "formally verified" does not mean "bug-free" and that they did not verify every entry point of every library.
• The same day, METR published a pilot stress test of the system OpenAI uses to monitor its internal agents. It wrote down what its report covers, what OpenAI redacted, and which OpenAI claims it did not evaluate.
• A word like "verified" says someone looked. To lean on it, you need three answers in writing from whoever said it: what exactly was checked, what it was compared against, and what was left out and who says so.
• If the claim comes in one language and your work is in another, there is a fourth question: was it checked in both?
• We put the three questions to three "approved" stamps from our own week. In eight of nine answers, the body of the report already had them. The label did not.
• You leave with a ready-to-send message that asks only for the answers you are missing.

The short answer. "Verified" tells you someone looked. It doesn't tell you what they looked at, what they compared it to, or what they left out. Before you lean on the word, get three things in writing from whoever said it: what exactly was checked, what it was compared against, and what was left out and who says so. If one is missing, you have a label, not a check, and the next move is to ask.

Picture an email in your inbox on a Tuesday. The subject line says "Verified and ready to go." There's a green check next to it. There's no attachment, no link, and no second sentence.

Most of us have received that email, or a cousin of it: a "reviewed" in a status update, a "certified" on a vendor's website, a teammate's "I already looked at it." The word is built to be comfortable. It fits anywhere and weighs nothing, which is exactly why it is worth weighing.

Monday's roundup covered today's two stories as two short facts. This piece keeps what didn't fit there: what you can ask when a word like that lands on your desk, and how to ask it without sounding like you're filing a complaint. One disclosure first: we work with Claude, made by Anthropic, one of the two organizations we discuss today. We are not a neutral third party, so we quote their page as written.

1. Three questions, always the same three

A claim is a sentence someone hands you so you can build on it. "Verified." "Reviewed." "Certified." "Tested." "Approved." "Safe."

It sounds like a measurement. A measurement has a subject and a yardstick: this thing, measured against that. And it has an edge: the part nobody measured. The bare word carries none of the three.

So you ask:

1. What exactly was checked?
2. What was it compared against?
3. What was left out, and who says so?

The answers matter, not the word

If the person making the claim wrote none of the answers, you don't have a check that is partly done. You have a label.

The answers have to come from whoever made the claim. If you fill them in with what you assume, you haven't read the claim. You've finished writing it for them.

2. Question one has a good answer when it comes with names and numbers

Here is what a written answer to question one looks like. On October 22, Anthropic published research done with Theorem on formally verifying code. The method turns code into a mathematical model and tries to prove it always follows a defined set of safety rules. The page says it covered 239 entry points across 123 open-source libraries.

It also says where that stops. The proofs hold for inputs under 64 MB, and they assume a single-threaded program compiled as a static binary, running on a RISC-V processor with Linux and the musl library. The page points out that most people run these libraries on x86 or ARM, linked against glibc.

And it adds three things that change how the number reads. The bugs it found were patched, and the patched code was proved again. Only a subset of the proofs is published, because they are not releasing proofs that could reveal a bug that has not been fixed yet. And what you have to take on trust is three things: the formal models of RISC-V and of Linux processes, the exact statements of the theorems, and the Rocq checker.

With an answer like that, you can go see whether your case falls inside or outside

Imagine the scheduling software your bakery uses sends a note: "Verified and ready for the holiday rush." You can believe it or not. That is all you can do. Now imagine the note names the number of orders per hour it was tested at, the devices it ran on, and the one thing it didn't test, which is your payment processor. You can open your own setup and compare.

None of this says that kind of verification applies to a small business, or that yours should look like it. It is simply a case where question one has an answer on the page, so it shows what an answer looks like.

3. Question two: something checked against your yardstick, or against theirs

A check is only as good as the thing it was checked against. "Against what?" is easy to skip, because the word sounds finished.

Anthropic's page says the specifications came from automatically converting the test suites each library already had. Claude writes the proof, and a separate checker, Rocq, confirms it satisfies the spec. The page also says that a proof checker "can't tell you whether your spec matches reality."

A check done against the wrong thing is still a check

That isn't a criticism of Anthropic. It's what the page itself says, and it's question two in its cleanest form. Take an event producer who tells you the setup is "approved." Approved against the room's fire-code capacity? Against the requirements in your contract? Against what they usually do? All three are comparisons, and only one of them is yours.

What to do: if the answer is "against our internal standards," ask to see them, or ask for a source you can open. And when they name a source, open it. A source appearing in the answer doesn't make it true, or true for you.

4. Question three: the edge, and whose hand wrote it

Every check has an edge. What changes from one claim to the next is whether anyone wrote it down.

Both of today's pages do, in different ways. Anthropic writes about its own work: "formally verified" doesn't mean "bug-free," and not every entry point was verified.

METR writes about someone else's. Its exercise was a pilot, high-access stress test of OpenAI's internal system for monitoring its agents for misalignment. It ran from July 21 to August 11, with access to the codebase, the monitor prompts and configuration, the documentation, and about a dozen interviews. Its findings describe the system "on and before July 21st," and it did not evaluate changes that were in progress. It also says OpenAI could redact anything it considered confidential or sensitive, in its sole discretion, and that a redacted 38-page version went to the leadership teams at METR and Redwood. About what OpenAI says it has fixed, METR writes: "We have not evaluated these claims."

An edge written by an outsider weighs differently than one written by the claimant

We aren't ranking one page above the other, and we aren't judging either organization. In one case you get the edge of a team's own work. In the other you get the edge of a team's view into someone else's work, and what that team did not touch. The tool that comes with the guide doesn't sort them. It lets you know which one you have.

And this holds for these two pages only. We don't know what other pages do, and generalizing from two would be doing the thing we're warning about: making a claim without saying against what.

A different question, who should review the work of whoever built it, lives in the people who built it can't be the only ones grading it. That piece asks for an independent reviewer with real access. This one reviews nothing. It reads what someone already told you.

5. What happened when we asked our own "approved" stamps

Before asking you to do something, we did it. We took three "approved" stamps from our own week, from this very house: the final review on Monday's piece, the style edit on that same piece, and the final review on Tuesday's. Then we put the three questions to each.

A note on how small this is. The person who wrote this piece did it, by reading each report. That is not an independent review, and we did not reopen any source. So it measures what each report says, not whether what it checked was true. We picked those three with no sampling method, and other reports from the week, including the independent red-team ones, were left out. Three reports over two days is not a sample, and we are not drawing a percentage from it.

What we found:

• The answers were there. They just weren't on the label. Three reports times three questions is nine answers. In eight, the body of the report already said what was opened, what it was compared to, or what was still pending. In one it didn't: the style edit never says it did not check whether the facts were true. The label ("PASS," "SHIP") said less than the body.
• Two checks contradicted each other on the same line. A style edit changed a heading to match what its list asked for, and another check, which compares against a different rule, said that heading was wrong. A perfect score from one lived next to a failure from the other, because each was comparing against something different.

Even our own "approved" doesn't say against what, on its own

If we wrote the label for those three reports, it would be "approved." If we wrote the full claim, it would be a longer and more awkward sentence. That is the sentence we're asking you to request.

6. When the claim speaks one language and your customers speak another

A claim can arrive in one language while your work happens in another. That opens a fourth question: was the same thing checked in both?

Imagine a clinic with a Spanish-speaking front desk that buys an intake form tool. The vendor's report says "tested," in English. Tested in which language? What helps is an answer like "we tested with Spanish text and screen-reader output, and here are the cases." What doesn't help is "it's a multilingual product," because that describes what the product can do, not what was checked.

If the claim names one language, yours is outside the check until someone says otherwise

A form, an interface, or a legal notice can work in one language and break in another. What matters is that your language appears in the answer, in writing. And ask in the language of the claim: if it came in English, ask in English.

The conversation about the same word, in two languages

Naty: I move between Spanish and English all day, and "verified" doesn't travel the same way in both. In Spanish I hear "verificado" and I think someone confirmed a fact. In English it can mean anything from "we ran a script" to "a person read it."

Todd: That's the part I can't see from where I sit. I work in English, and when a vendor says "verified" I tend to read it as done. Naty is the one who hears the gap, because she carries the Spanish side of our work.

Naty: And then we flip it onto ourselves. If I write "reviewed" in a Spanish piece and Todd writes it in an English one, are we saying the same thing?

Todd: We weren't, at first. Now the edge ships with the work: whatever we call reviewed, we also write down what we didn't open.

Naty: It changed how I read my own checkmarks. I asked it of our own reports this week, and the answer is in section 5.

This week, one task: your message

Fifteen minutes, one claim.

1. Pick one. The latest "verified," "reviewed," or "certified" that reached you this week, from a vendor, a contractor, a bank, a seal, or your own team.
2. Answer the three questions using only what they handed you, not what you assume: what was checked, against what, and what was left out and who says so. If the claim is in one language and your work in another, add the fourth.
3. Send in writing only the questions that have no answer. Open with one sentence about why you want them ("before I rely on this, I need these answers in writing") and close with one that lets them say what was not checked. If every question had an answer, there is nothing to send. What's left is deciding whether what they left out is something you can live with.

The guide has the rest: the message line by line, how to read what comes back (the answer that names things, the one that repeats the word, and silence), and what to do with whatever was left out. The tool that comes with it builds the message for you, with just the questions you're missing.

Text the word on WhatsApp and both reach you there.

Frequently asked questions

What does it mean when a vendor says something is "verified"? It means someone ran some kind of check. It doesn't say which one, against what, or what was left out. To lean on the word, you need those three answers written by whoever said it.

Will asking what they didn't check come across as distrust? It can, if it's sent as a complaint. It lands better with one sentence about why you want the answer ("before I rely on this") and a closing line that makes it easy for them to say what wasn't covered.

What if they just answer, "Yes, it's verified"? They answered a question you didn't ask. That can be a habit rather than evasion: the person may not have the details at hand. Ask again about one thing only, the one that matters most, in the same words.

Is a vendor that names what it didn't check more trustworthy? We don't know, and we're not claiming it. What changes is what you hold: with the edge written down, you can decide how much to lean on it. Without it, you have to guess.

The claim is in English and my customers speak Spanish. What do I ask? Whether the check was done in both languages. If the answer names only one, yours is outside the check until they say otherwise. Ask in the language of the claim.

Closing

The two pages that helped us most this week have something in common: they write down where their work stops. That doesn't make them right, and it doesn't tell us their claims are true. It tells us what to read them against.

Your version of that fits in a WhatsApp message: one claim, three questions, and, if one is missing, a short note asking for it. The word says someone looked. What's useful is what they looked at.

Thanks for reading to the end. If one of your three answers came back empty, we'd like to know which one: we both read what you send.

Keep reading

• Anthropic and the banks said who approves what AI does - Monday's roundup, where these two pages show up as two short facts
• The people who built it can't be the only ones grading it - who should review, which is a different question
• Your AI hands you a lot. Count how much of it you use - last week's guide and tool
• SEO vs. GEO vs. AEO: which one does your business need in 2026? - the full map
• LLMs are making formal software verification feasible at a much larger scale, Anthropic - the full page

Sources

• LLMs are making formal software verification feasible at a much larger scale, Anthropic, October 22, 2026 (modified October 23) - https://www.anthropic.com/research/formal-verification
• Red-Teaming OpenAI's Internal Agent Monitoring Systems, METR, October 22, 2026 - https://metr.org/blog/2026-10-22-red-teaming-openai-agent-monitoring/