Imagine six objects, five rules, and one correct arrangement. You place them in order, submit your answer, and learn only how many objects landed in the right spots—not which ones. You can keep working, or ask an AI for a clue: the correct position of one randomly selected object. Every clue is right. But each one costs points.
That setup turns a familiar promise of artificial intelligence into a puzzle of its own. If help makes a task easier now, does it also make the solver better at the task later? Or can someone look successful with an assistant at hand while learning less about how to solve the problem alone?
To investigate, researchers recruited 150 adults in the United States, then analyzed results from 124 participants who passed the study’s quality checks. Participants had at least an undergraduate degree. Each person solved logic puzzles in three stages: an initial round without AI, a 20-minute middle round with or without access to AI, and a final round without AI. The first and final assessments lasted eight minutes or until participants had completed at least four problems.
In the middle round, the researchers randomly assigned participants to one of three conditions. Some had no AI access. Others could request a clue, with either 0.1 or 0.18 points deducted for each request. The assistant did not reveal the whole solution or explain how to reason through the clues; it supplied one correct object position. For this experiment, it was simulated and perfectly accurate, allowing the researchers to study help-seeking without accuracy changing from request to request.
The first clue came from people who asked for no help at all. Among these 75 participants, the rate of solving objects correctly rose from 2.03 per minute in the first assessment to 3.86 in the final one—a 90.2% increase. Practice alone, it seemed, could sharpen performance. But that did not yet reveal whether asking for AI help changed what participants learned. People chose whether to request clues, and those who did might have differed from people who didn’t before the experiment even began.
A second clue came from comparing the final, unaided round. Participants who had requested AI help averaged 3.42 correct objects per minute; those who had not averaged 3.86. And a model using performance during the middle round to predict later performance tended to overshoot for AI users: it predicted their unaided results 0.22 reward-rate units too high on average. For participants who never requested help, it underestimated final performance by 0.15 units.
That is a suggestive mismatch, not proof that the clues caused a loss of skill. To look beyond raw scores, the researchers built a Bayesian model that treated accuracy and response time as imperfect clues to an underlying ability. The model estimated how participants’ ability changed between the first and last assessments, while accounting for their starting ability. It also tracked something more subtle than the number of AI requests: the share of middle-round puzzle time participants spent thinking independently, before or without asking for help.
Here the trail took an unexpected turn. The count of AI requests, by itself, was not associated with estimated skill change once starting ability was accounted for. The researchers’ alternative model estimated the relationship between request frequency and latent skill change near zero, with a 95% credible interval that included both negative and positive values. Simply asking for help more often did not explain who improved.
Time spent reasoning independently did. Participants with a greater “solo share” tended to show larger estimated gains. In a model-based comparison, one additional minute of independent reasoning during the 20-minute AI-access phase corresponded to 1.5% higher predicted accuracy and 2.5% lower predicted response time in the final round, compared with an otherwise average participant. Those figures describe an association estimated by the model, not a guaranteed payoff from any particular minute spent thinking.
That distinction matters because the cost of a clue did influence behavior, though the evidence was modest: people in the low-cost condition averaged 6.67 requests, compared with 3.33 in the high-cost condition, a difference reported at p < 0.10. Yet the model’s sharper signal was not how often people clicked for help. It was how much puzzle-solving they continued to do themselves.
There are still rival explanations. Independent effort was observed, not randomly assigned. People who persisted longer might have had other qualities that helped them learn, so the study cannot establish that preserving solo time caused better later performance. The task was also narrow and short, and the assistant gave guaranteed-correct position clues rather than the varied answers and explanations people encounter in everyday AI use. The researchers describe their findings as short-term and task-specific, and call for testing in larger samples, over longer periods, and in more realistic settings.
So the result is not that using AI inevitably makes people worse at thinking. Participants improved overall, and request frequency alone did not account for who learned. The more intriguing possibility is that help has different effects depending on what it replaces. A clue can be a shortcut around the reasoning—or a small nudge inside a problem the learner is still working through. The unanswered design question is how to make AI assistance useful without quietly taking over the practice that builds skill.
For now, the experiment offers a practical lead rather than a final verdict: an AI tutor might ask a learner to try first, offer a partial hint instead of an answer, or invite reflection after a clue. Whether those designs preserve learning remains to be tested. In this puzzle, the best clue about future skill was not how much help someone received, but how much thinking they kept for themselves.
