Take a long explanation for a hard math problem. Cut it into lines, shuffle them, and black out the words. You might expect the solution to become unreadable. In one experiment, a language model could still retrieve the correct answer from a trace transformed that way in 83.3% of cases.
That result sounds like a challenge to the idea that AI reasoning unfolds step by step. But the researchers’ experiment had a more careful—and more intriguing—target: not whether a model could solve a problem from scratch, but what it could extract from an explanation after it had already been written.
The team tested three language models on math, coding, and science questions. First, each model produced a reasoning trace: the working that comes before its final answer. Then the researchers altered that trace, making versions with scrambled lines, missing words, masked digits, or added false answers. The same model was asked to produce an answer immediately from each altered version.
That last step mattered. In a free-form response, a model might ignore a damaged explanation and try solving the problem again. To isolate extraction instead, the researchers gave it an answer-completion cue and measured whether it could retrieve the answer from the provided trace. The experiment was a controlled inspection of the trace’s contents, not a new way of generating reasoning.
The first surprise came from the order of the lines. On the AIME 2025 math problems, shuffling the lines barely changed retrieval accuracy for the three models. One model stayed at 78.00%; another moved from 91.33% to 90.94%; and the third went from 88.67% to 89.67%. The steps no longer appeared in the order they were written, but the answer remained recoverable.
Next came the scissors. In one AIME test, the researchers removed every alphabetic character from a model’s trace, leaving digits, symbols, and spacing, and then shuffled the lines. GPT-OSS-120B retrieved the correct answer in 83.3% of cases, compared with 91.3% when given the original trace. Even masking the prose rather than deleting it improved accuracy by 4.7 percentage points in a separate condition.
It is tempting to picture the model reading a jumbled handful of numbers and independently reconstructing the proof. The experiment does not establish that. The math question was still part of the prompt in the 83.3% test, and the transformed trace still contained the answer’s occurrence as well as the numerical work around it. In other words, the explanation had been stripped down, but its important clues had not all been removed.
The researchers checked that distinction directly. Masking the answer in an otherwise intact trace lowered GPT-OSS-120B’s accuracy from 91.3% to 73.3%. When they removed alphabetic characters, shuffled the lines, and removed the answer too, accuracy fell to 46.7%. Masking the digits as well as the answer drove it to 0%. The prose was often dispensable; the numerical material was not. And the answer itself acted as a useful anchor.
There was another twist. The team added a false answer repeatedly—up to three times as often as the true answer appeared in the trace. In the stripped-and-shuffled condition, accuracy stayed at 83.3%. That suggests the model was not simply picking the most frequently repeated number. The researchers propose that relationships among the numbers may help constrain the answer, but they emphasize that this explanation remains a conjecture, not a demonstrated account of the model’s internal workings.
The pattern also had limits. The strongest tests of how much information could be removed were concentrated on AIME’s math problems, where numerical notation is central to the task. Performance varied across the coding benchmark, and some forms of shuffling caused large drops there. Word-level scrambling, too, was less dependable than moving whole lines. A result about finding information in reordered lines is not a license to assume that order never matters.
So what, exactly, survived the experiment? Not proof that AI systems think in unordered piles of symbols. The researchers did not inspect the model’s internal mechanisms, nor did they show that a model can generate its reasoning in parallel or safely discard most of it. They tested a narrower stage: whether a model could retrieve an answer from its own altered trace. That distinction turns the headline-grabbing result into a useful scientific question about what an explanation contains—and which parts a model actually uses.
The practical possibility is still enticing. If future work finds ways to generate shorter traces without losing the relationships needed to answer correctly, reasoning systems might spend fewer tokens or produce some intermediate steps in parallel. But this study did not achieve that. Its unexpected result is a clue, not a shortcut: a long explanation can look like a carefully ordered argument to a reader, while a model’s answer-retrieval process may lean on a smaller set of clues embedded inside it.
