First Proof project calls upon UB expert to test AI’s prowess in math

Adam Sikora, professor in the Department of Mathematics, College of Arts and Sciences, developed a research-level math problem recently used by the First Proof project to test artificial intelligence’s evolving mathematical capabilities.

First Proof is an independent experiment led by Harvard University and others that assesses advanced AI systems’ ability to solve complex mathematical problems and explain how they arrived at their solutions. Organizers collect 10 problems from mathematicians who have already solved them but have not yet published their solutions, and then challenge leading AI models to solve them independently within 24 hours.

Sikora was invited to submit a problem for what First Proof calls its “third batch”Download pdf(530 KB) of experiments, whose results were released Wednesday.

“Given the caliber of the people involved and the importance of what they are trying to measure — whether current AI systems can genuinely reason about difficult, previously unseen mathematics — I was honored to participate,” Sikora says. 

Sikora’s problem came from his ongoing work with his collaborator Thang T.Q. Lê, professor of mathematics at Georgia Institute of Technology. Related to the branch of mathematics known as quantum topology, the problem asks whether a complicated abstract mathematical object called a quantum group can be described entirely through pictures: diagrams drawn on a two-dimensional surface.

“One way to think about it is as an attempt to turn a difficult algebraic object into something geometric and visual,” Sikora says. “Mathematicians often gain new understanding when the same structure can be viewed from two very different perspectives, in this case, algebraically and geometrically.”

According to First Proof, only one of the five AI systems tested successfully solved Sikora’s quantum topology problem. ProofCouncil produced a solution that was determined to be mathematically correct, clearly explained and properly cited previous research.

Another system, OpenAI’s GPT-6 Astra, developed a largely correct approach but was found to have left gaps in its proof. The other three systems either produced only partial results, submitted no solution or made significant mathematical errors.

Sikora was impressed by ProofCouncil’s solution, but says it’s worth noting that it is a more sophisticated and expensive system that coordinates several leading AI models together, and requires technical programming expertise as well as paid access to multiple high-end AI models. 

“At present, that makes this kind of setup much less accessible to most academic researchers,” he says.

Overall, ProofCouncil solved nine of the 10 problems. However, organizers noted that even the leading systems struggled to clearly explain their work and properly cite existing research. They also found a significant performance gap between the leading systems and the one system built on an open-weight model, which anyone can download and run.

The higher computational costs associated with better AI performance also raised concerns that access to these resources could become a barrier to mathematical research.

This latest experiment comes as the field of mathematics grapples with rapid advances in AI. Last month, OpenAI announced that its AI agents had solved one of the remaining Millennium Prize Problems, a set of mathematical challenges identified in 2000 as among the field’s most difficult. 

And just last week, the company released hundreds of AI-generated papers reporting progress on other longstanding problems, including another Millennium Prize Problem: the 166-year-old Riemann hypothesis.

Sikora says that while this is certainly a “watershed moment” for mathematics, AI systems are still not universally better than human mathematicians. He also notes the models’ ongoing issues with explanations and citations. 

“The AI revolution is certainly disruptive to mathematics. It forces us to rethink the nature of mathematical research, how we train PhD students, how authorship and intellectual credit should work, and even how publishing and peer review can function if producing mathematical results becomes much faster. We do not yet have answers to many of these questions,” Sikora says.

“Nevertheless, I am cautiously optimistic. I think we are in a process of developing a new and very powerful culture of mathematical research in which humans continue to play the leading role — choosing important questions, developing concepts, judging significance and seeking understanding — while being greatly empowered by AI,” he adds.

“If we manage that transition well, the result could be an extraordinary acceleration of mathematical discovery and, ultimately, substantial benefits for science and society.”