OpenAI's 722 AI-written math papers draw scrutiny; three withdrawn for errors

OpenAI has released 722 mathematics manuscripts written by an internal artificial intelligence model that has not been made public and said they contain new results on hundreds of unsolved problems, but mathematicians say much of the output remains unverified and three manuscripts have already been withdrawn. According to Newsis, citing Decrypt on 8 October local time, the papers were posted on GitHub and are organised into 372 groups of related results, so 722 papers does not mean 722 problems solved. OpenAI said it gave the model about 4,000 problems and published only the results it judged meaningful; the problems span fields including algebra and mathematical logic. [ 1 , 2 ]

ZDNet Korea, citing TechCrunch, reported that OpenAI released the manuscripts on 6 October and withdrew three within a day after errors were found. Chosun Ilbo reported that one manuscript needed a particular value to equal zero for the next step of the proof but applied a plus or minus sign incorrectly, so that the required condition was not met; two other manuscripts that depended on that calculation were withdrawn with it. OpenAI also revised 14 manuscripts to supplement proofs or correct explanations. The company said it found the errors in its own checks and would fix them quickly or withdraw what could not be fixed. [ 2 , 3 ]

OpenAI used Lean, a program that mechanically checks each logical step of a proof, to verify part of the results, and the outlets give different figures for how much. Newsis reported that the core result was confirmed with Lean in 162 of the 722 papers, about 22%. ZDNet Korea reported that computer verification material existed for 300 of 719 manuscripts, about 42%, and that even the machine-checked work needs further confirmation by mathematicians. OpenAI acknowledged that not all results are verified and that some results that have not been formalised may contain problems. [ 1 , 2 ]

Researchers at the University of Cambridge and King's College London examined the Navier-Stokes work OpenAI said last month it had solved and found that the written solution and the Lean verification code differ in at least two places. According to Chosun Ilbo, the code checking one intermediate calculation carried stronger conditions than the manuscript, which claimed the calculation held under fewer conditions, so passing the check does not mean the written explanation was verified. The researchers said this alone does not show the final Navier-Stokes conclusion is wrong, and that manuscript has not been withdrawn. [ 2 , 3 ]

OpenAI said most of the results came from a single AI agent given one prompt, with some requiring several attempts, and that each result used on average about three hours of ChatGPT Pro inference compute, far less than the 10,000 agents run for 88 hours on Navier-Stokes last month. Andrew Sutherland, a professor at MIT, said the single-agent claim should be treated as unverified until the model is released and other researchers reproduce the results. OpenAI did not release the prompts, although an advisory group at the Institute for Advanced Study in Princeton recommended last month that model name, prompts, reasoning summaries, time and compute cost accompany AI math results; OpenAI published average compute and reasoning summaries for 10 studies. [ 1 , 2 ]

The IAS advisory group called the release an important event for mathematics but said that having advised OpenAI did not mean it endorsed the results or how they were announced, and it has opposed AI companies solving problems with models outsiders cannot use. Mathematician Terence Tao criticised cases where people claiming AI had solved a problem could not answer other mathematicians' questions or explain the work at conferences. Melanie Wood, a Harvard mathematics professor, said that when a model is told to solve a hard problem, people do not understand the result at the time it is announced and the real work begins then. Professor Abhishek Saha called it a very important day for mathematics but said most of the results were advances within existing lines of research; Newsis reported that only one problem, the Quasi-Riemann Hypothesis, was said to approach Millennium Prize level. OpenAI said it would publish further Lean verification, consider releasing the model, and support workshops and conferences so mathematicians can review the results. [ 1 , 2 ]