Claude Generates 180 Million Lines of Proofs to Validate Software

Anthropic’s Claude model recently generated approximately 180 million lines of mathematical proof code to verify the correctness of 123 open-source software libraries. This achievement marks a significant shift in how software reliability can be assessed, moving beyond traditional testing methods toward automated formal verification. The project was conducted by Anthropic’s Frontier Red Team in collaboration with partner Theorem over a period of just a few weeks.

Formal verification has historically been too expensive for widespread use. Specialized teams once required twenty person-years to verify the small operating system kernel known as seL4. Consequently, the technique was largely reserved for high-stakes environments such as Airbus avionics. By leveraging Claude, the researchers translated crash tests from Google’s OSS-Fuzz security project into mathematical problems. The model then analyzed compiled machine code to produce extensive proofs, a process that required minimal human review.

The system relies on an independent verification program called Rocq to automatically check the generated proofs. While the formal foundations and tools were built by humans at Theorem, the actual proof generation was described as fully autonomous. This approach allows for the verification of existing software rather than writing new code from scratch. The method addresses a core challenge in AI-generated software by providing mathematical guarantees of correctness under specific conditions.

During the verification process, Claude identified hundreds of bugs that conventional testing had missed. Many of these issues only surface under very specific conditions, such as unusually large input files or particular sequences of access. In one notable instance involving a widely used PDF reader, the team found as many security bugs in a single week as are typically reported for that software in an entire year. Claude also corrected a logical error in another application that had remained undetected for more than a decade.

Anthropic notes that even when a complete proof fails, the attempt often surfaces real bugs. This suggests that the verification process has value beyond just producing a successful certificate. The model was allowed to fix the bugs it found, demonstrating a cycle of detection and repair. This capability is particularly useful for identifying edge cases that standard test suites might overlook due to their rarity or complexity.

However, the term formally verified does not mean the software is entirely error-free. The proofs hold only under strict conditions, including a specific processor architecture, single-threaded execution, and inputs under 64 MB. In practical use, these libraries often run on different hardware and configurations. Additionally, the compute cost is significant, with individual proofs consuming up to hundreds of millions of tokens. Anthropic did not report the total project costs, but the efficiency gains from newer models are notable. Opus 5.5, for example, solves half the proofs with about 40 percent fewer tokens than its predecessor, Opus 5.

The project strengthens the case for broader software automation. If AI can both write code and guarantee its safety under defined conditions, the industry can trust AI-generated software more readily. This could reduce the time spent on manual code review and lower overall development costs. The implications extend beyond current coding tools, potentially transforming how critical infrastructure is maintained and secured.

Anthropic expects formal verification with AI to become as routine as standard testing in the near future. Theorem’s roadmap already targets critical infrastructure components such as glibc, OpenSSL, and cURL. As newer models become more efficient, the economic barrier to entry for formal verification continues to fall. This trend suggests that mathematical proof generation will become an integral part of the software development lifecycle, ensuring higher standards of safety and reliability for everyday applications.