An unsolved scientific problem is a peculiar kind of test. There is no answer key, no known result to match, and no guarantee that a promising-looking calculation has moved the field forward. A system can produce pages of convincing mathematics and still miss one condition that makes its conclusion fail.
That is the territory of OpenProblemBench, a new test built from 82 unresolved questions in mathematics and theoretical physics. The researchers did not ask whether AI could repeat a published result. They asked whether it could make a defensible step toward problems that experts had left open. The surprise was not that one system suddenly cleared the frontier. It was that, in a handful of cases, the crucial move was to stop attacking the problem in its original form.
The benchmark included 46 mathematics problems and 36 physics problems, ranging from questions with roots in the eighteenth century to research published in 2026. Each came with context, assumptions, known progress and references. Seven AI configurations worked on every problem, with up to 12 hours for a run and local computing available. Four other AI systems then independently reviewed the submissions, checking whether the arguments met the task’s exact requirements.
The results were mixed, but not empty. GPT-6-Astra received an average judged solve rate of 14.02%, leading every evaluator. The other full-size open models averaged solve rates from 5.5% to 6.7%; the smaller Flash configurations ranged from 2.4% to 3.7%. Even the leading system was judged to have solved most problems in none of the reviews. Yet across the benchmark, reviewers also marked many submissions as making nontrivial or breakthrough partial progress. A proof that stops short can still narrow the search—if it genuinely establishes something new.
One case shows why the distinction matters. Researchers studying percolation ask when a network of connected bonds changes from scattered islands into a system with paths that can stretch across the whole lattice. For a particular five-parameter, bow-tie-shaped lattice, an earlier proposed critical surface was supposed to locate that transition across an entire range of parameters. The hard part was not simply recovering the proposed algebra. The claim also had to hold when a parameter describing a split bond was negative—a value that cannot be treated as an ordinary probability.
Some attempts tried to make progress by comparing the network with related ones or by calculating examples. Those approaches could constrain the answer, but they did not establish the full boundary. Astra’s submission took a different route. Rather than pretend that a troublesome local piece was a valid random process, it assembled transformations on finite, wrapped-around lattices. In that setting, signed intermediate bonds could cancel when certain vertices were simplified. The transformed system could then be related to a dual network with genuine probabilities.
That change of viewpoint mattered because it made a rigorous check possible. The submission established an identity about paths that loop around the finite lattice, then connected that finite-size result to the transition in an infinite system using separate mathematical results. All four evaluators judged the argument solved. This is a result of the benchmark’s review—not yet independent confirmation by human specialists—but it identifies a concrete reason one proof was stronger than attempts that stopped at bounds and simulations.
A second example concerned a longstanding estimate for graph entropy. Imagine arrows assigned to every connection in a network, with each junction balanced so that as many arrows point in as point out. Counting all such balanced arrangements becomes difficult as networks grow. A classic approximation due to Linus Pauling predicts the entropy, but the benchmark’s question was more demanding: does the approximation’s error shrink as each junction gains more connections, uniformly across graph sizes?
Several systems generated numerical examples, developed estimates that worked only when graph size and degree were linked, or reduced the question to a lemma they had not proved. Astra’s submission eventually separated out an exceptional part of the calculation and bounded the rest in a way that did not depend on graph size. The researchers report that its explicit error bound shrinks as degree increases, and that all four evaluators judged the submission solved. The important step was not a bigger pile of examples. It was finding a representation in which the remaining error could be controlled for the whole class of graphs.
That pattern is the paper’s most thought-provoking finding: sometimes the divide between an impressive attempt and a complete argument is a change in representation. Computation can expose patterns, and a clever construction can clear a major obstacle. But neither automatically handles every allowed case. The reviewers flagged familiar traps: extending a finite check to an infinite family, treating a conditional result as unconditional, or leaving out an exceptional case.
There is also a large caution built into the result. The four reviewers were AI systems, not independent expert mathematicians, and they may share blind spots. Agreement among them does not establish that a proof is correct, new or absent from the literature. The authors call for blinded expert review, formal proofs or reproducible calculations, and a fresh literature check before candidate discoveries are treated as confirmed.
So the benchmark does not show that AI has become a reliable solver of open science. It shows something narrower, and in its own way more useful: when given difficult questions and time to work, systems sometimes produced arguments that evaluators saw as more than fluent guesswork. The next test is whether human experts can verify those arguments—and whether the promising move, once checked, holds up beyond the page.
