Picture a small Sudoku-style grid. You are asked to place a particular digit in any empty cell where it is legal. There may be several. Choose one, and you are right. Now ask the same question a hundred times—not of one solver, but of a crowd. Do the answers spread across the possibilities, or pile up around a favorite?
That is the puzzle behind a study by Zihao Wang, Francesco Insulla, and Andrea Montanari. Instead of asking only whether people and AI could solve a problem, they asked how each group chose when the problem allowed several equally valid answers. The distinction matters: a scorecard can call two solvers equally correct even if one explores a wider range of options.
The researchers built 270 small puzzles, each with three to eight correct answers. They included arithmetic challenges, mazes, placing rooks without conflicts, Minesweeper, and Sudoku. In the main set, there were 100 puzzles, 20 from each family. Across the study, 521 U.S. adults took part; 104 were retained for the main module. Three AI models were each asked to respond repeatedly to the puzzles, with the primary comparison using a plain prompt and low reasoning effort.
The first clue emerged in the answer distributions. The researchers measured how broadly each solver spread its correct responses: a score near zero meant repeatedly choosing the same option, while a score near one meant answers were spread almost evenly across the possibilities. Averaged across the five puzzle families, people scored 0.82. The three models scored 0.29, 0.48, and 0.45.
Sudoku made the contrast especially sharp. Human answers came close to an even spread, with a diversity score of 0.91. One model’s score was just 0.02: its valid answers were concentrated almost entirely on the same choice. The other two models spread their choices more widely, but still less than people did. These comparisons count correct human responses and valid model answers, so the pattern is not simply a matter of counting mistakes as different strategies.
But perhaps the models were just less capable at the puzzles. That explanation sounds plausible—until you notice what the experiment was designed to compare. When several answers are valid, correctness cannot tell us which one a solver tends to prefer. The finding is not that people solved every puzzle better. It is that, among acceptable answers, their choices were more varied. The researchers’ measures expose a difference that a conventional right-or-wrong score would hide.
Then came a second clue: keep the puzzle’s answer set the same, but change its presentation. The team highlighted cells or reflected boards, mapping answers back to their original positions. These visual changes shifted the models’ answer distributions substantially; people were less affected. That does not reveal exactly what the models were attending to, but it shows that identical underlying choices need not produce identical patterns of selection when the display changes.
People, meanwhile, showed a different kind of sensitivity: what puzzle they had just seen. When a puzzle followed a related one, human answers to the pair were more connected than the models’ answers were. The researchers also tried a strategy primer—a separate puzzle with a simple solution pattern that could match one answer in the target puzzle. A shift toward that answer would be suggestive, but it would not prove people had transferred an abstract strategy: the match also involved visual or positional similarity.
Could a nudge make model choices more human-like? The researchers tried a prompt asking models to act as human participants, and compared low with medium reasoning effort. Neither change consistently brought their answer distributions closer to people’s. More effort was no simple cure for narrow choices, and its effects on diversity could differ from one puzzle family to another.
So what explains the gap? The study leaves that question open. It measures choices, not the private reasoning process that produced them. The authors suggest that different problem-solving strategies may be involved, but the experiments do not identify a single cause. And the evidence has boundaries: these were small, enumerable puzzles, a U.S.-based participant pool, three specific AI models, and a limited set of prompts and settings. A pattern in Sudoku or Minesweeper is not proof of how a model will choose in a medical, scientific, or everyday decision.
Still, the puzzle points to a useful change in what we ask of AI. If several answers meet the rules, accuracy alone leaves a question unanswered: which valid options does a system tend to favor, and how easily does that preference shift? A solver can be right again and again while exploring only a narrow corner of the answer space. To understand how people and AI differ, researchers may need to examine not just whether they reach the solution, but which of the right answers they choose along the way.
