The puzzle begins with a wooden panel and four moving parts. One is the goal: pull the leftmost joint to open the lockbox. But the joints constrain one another in a hidden pattern. A bar that moves now may block a handle later; the right sequence must be discovered through trial and error.
Researchers gave this mechanical puzzle to a robot controlled by a large language model, then changed what the model could see. Would it do best with an ordinary camera image, an image plus depth information, or a precise symbolic report of every joint’s state? The obvious answer is the most exact one. The experiment produced the opposite: the model reached the same top success rate with each input type, but got there faster with ordinary images—and more slowly with perfect state data.
That result opened a second puzzle. If the model did better when its observations were less reliable, was the visual information helping it reason, or was a little uncertainty disrupting a bad habit?
The physical setup made the challenge concrete. A Franka Emika Panda robotic arm, fitted with a soft hand and guided by a camera and force-torque sensor, manipulated the four joints on the wooden panel. Each joint had two possible positions. Which ones could move depended on the states of the others, but the robot was not told those dependencies. It had to infer the lock’s structure from what happened after each action.
For each move, the robot returned to a consistent viewing position, captured a new observation, and passed it to the model. The researchers tested three ways of presenting the scene: RGB images, RGB-D images that add depth, and a symbolic description of the joints’ actual states. The task was the same each time, and the agent had at most 20 interactions to pull the target joint. They ran 10 independent trials in each condition.
The model’s best physical performance was an 80 percent success rate in every input condition. But the route to that result differed. With ordinary RGB images, it reached 80 percent success after 11 steps; with ground-truth state descriptions, it took 15. The RGB-D condition fell between them at many intermediate points. A human-inspired strategy, included as a comparison, reached 100 percent success within nine steps. So the robot was not out-solving people. It was exposing a mismatch between the amount of information it received and how well it used that information.
One suspect was perception. The model sometimes misread whether a joint had changed position after an action. Such errors averaged 21.7 percent per trial with RGB, 26.8 percent with RGB-D, and 4.9 percent with ground-truth state input. But that clue did not solve the mystery. Even when given the correct symbolic state, the model still sometimes hallucinated a change that had not occurred. In that condition, the problem was no longer visual recognition; it was reasoning about the information in front of it.
Could the pictures have supplied useful clues that the state descriptions left out? The researchers considered that possibility, but their next test pointed toward another explanation: uncertainty itself might change the agent’s behavior. They moved to a controlled simulation, where a model received symbolic states, then deliberately altered what it was told about the result of an action. The simulated lockbox itself stayed unchanged; only the perceived outcome was flipped, at probabilities from 0 to 60 percent.
Here the performance curve was not a simple slide from good information to bad. With no flips, average success was 23.3 percent. Success rose as the flip probability increased, peaking at 40 percent—about 2.85 times the noise-free baseline—then fell as the observations became still less dependable. The errors had not made the puzzle easier. They had changed the model’s pattern of actions.
The researchers looked for a behavioral clue and found repeated action loops: sequences of three or more moves that appeared again within the same trial. Imagine an agent trying a few joints, returning to the same combination, and spending precious moves circling familiar ground. Loops were common when the model received perfectly accurate feedback. Their occurrence dropped as simulated noise increased, reaching its minimum near the same 40 percent flip probability at which success peaked, before rising again at higher noise levels.
Trials with more loops also tended to have lower success; the reported correlation was negative. That link makes the loop a plausible piece of the explanation, not a proven cause. The authors suggest that when an attempted move fails and the reported state remains unchanged, the model may receive no new signal to prompt a different strategy. A mistaken report can, oddly, break that sameness and push the agent away from a repeated sequence. But too many mistaken reports scramble the state enough to hurt again.
That is the puzzle’s most unsettling clue: a machine can appear more capable not because it understands more, but because an error interrupts a failure pattern. The study does not show that noise is generally useful, or that robots should be given deliberately faulty sensors. The simulation used a different model from the physical experiment, and the researchers tested a single lockbox layout and models from one provider. They explicitly treat the simulation as evidence for a possible mechanism, not a direct explanation of the physical results.
What the lockbox does reveal is how hard it can be to judge a problem-solving system by its final answer alone. A robot may succeed through sound inference, repeated luck, or a quirk in the way its observations steer its next move. To tell those apart, researchers need to watch not only whether the robot opens the box, but how it gets there—and what happens when one piece of the feedback loop changes.
