Anthropic is developing FlashProve, an open-source system to prove which AI model produced an output
The "provable inference" approach would let users cryptographically check that an answer came from the model they were promised, even if the lab running it is the one cheating.
In a blog post published Tuesday, Anthropic described FlashProve, an open-source system it is working on to verify which AI model generated a given output. The goal is what researchers call "provable inference": a cryptographic proof that a specific output came from a specific model. TechCrunch compares the idea to email signatures that let a recipient confirm who sent a message. The system is meant to resist impostors who disguise one model as another, including when the impostor is the lab that built and runs the model. The researchers say it could stop an AI company from quietly swapping in a cheaper model to save money, or a more expensive one to inflate benchmark results. It could also let third parties check claims about AI-performed work. "A developer might claim that their software library was scanned for vulnerabilities by a frontier cybersecurity model, or a company might claim that its business plan was given the go-ahead by an appropriate model," the researchers write. "How can a user verify these claims?" For the system to work, AI labs would have to publish a "fingerprint" of each model they release, which could later be matched cryptographically to specific outputs. How useful it turns out to be will depend on labs agreeing to publish these fingerprints. If they do, API customers, auditors and regulators would have a way to check claims they currently have to take on trust.