Imagine a robot reaching for a mug. Before its gripper moves, its internal model can run a preview: the arm bends, the mug lifts, the hand carries it toward a shelf. That preview might look perfectly plausible. But if tiny errors creep into every predicted movement, the mug’s imagined path can slowly slide away from what the laws of motion allow.
That gradual drift is a problem for AI systems that learn to picture what actions will do. A robot’s world model is less like a video playing on a screen than a compact internal sketch: it turns observations into hidden states, advances those states in response to actions, then uses them to predict what comes next. The sketch doesn’t need to preserve every pixel. It does need to keep the important motion coherent.
A recent study takes that challenge seriously by borrowing an organizing idea from classical mechanics. The researchers’ method, called a Hamiltonian World Model, or HWM, gives a model’s hidden states an extra structure: one part plays the role of generalized coordinates, another the role of momentum. These are learned quantities inside the model, not measurements of the robot’s literal position or momentum.
That distinction is the doorway into the hidden world. In ordinary mechanics, coordinates describe a system’s configuration; momentum helps describe how its motion is carried forward. Together, they give a compact way to represent how a system changes. HWM builds an analogous space inside an AI model, then learns an energy landscape to guide how its internal state evolves.
The method does not try to replace the model that sees or renders a scene. Instead, it changes the machinery that carries a hidden state into the next moment. The state is mapped into the learned coordinate-and-momentum space, advanced using a rule shaped by Hamiltonian mechanics, and mapped back so the original model can make its prediction. The rule also includes ways to account for robot actions, energy loss such as friction, and leftover effects such as contact or deformation.
Why bring physics into a representation that isn’t itself a physical map? Because a prediction engine that only learns what usually comes next can be good at short clips and still lose its footing over longer ones. A small mismatch in the imagined movement of an arm or object can feed into the next prediction, and then the next. The researchers’ approach tries to give that chain a steadier internal rhythm, rather than asking a generic neural network to discover all the structure from scratch.
They first tested the idea in simulation, where the motion has known physical annotations. In one experiment, the models learned to predict 10 frames ahead and were then asked to roll their predictions forward for 60 frames. The HWM version reduced mean squared error from 62.32 to 53.03 and mean absolute error from 95.32 to 82.58. The researchers also measured more intricate features of movement: at a 30-frame rollout, acceleration error fell by 15.8% and jerk error by 19.3% compared with the unstructured baseline.
Jerk—the rate at which acceleration changes—is a reminder that motion has texture. A sequence can place an object roughly where it belongs yet still imagine it moving with the wrong start, stop, or jolt. By improving estimates of these higher-order details, HWM suggests that the hidden dynamics matter not only for where an object ends up, but for how it gets there.
The team then inserted HWM into several existing robot world models, changing the transition mechanism while leaving their perception and rendering components in place. On Ctrl-World, a model for single-arm manipulation viewed from multiple cameras, the reported nDTW score rose from 0.670 to 0.735, while its Logic score rose from 93.3% to 96.7%. On EVAC, which models two-arm interaction from a single view, the Logic score increased from 80.0% to 86.7%. These results point to better consistency in the predicted scenes and actions across those test settings.
Finally, the researchers asked whether a more stable imagined future could help a simulated robot complete tasks. With HWM added to Cosmos-Predict2.5-2B, average success on the LIBERO benchmark went from 97.8% to 98.7%; on RoboCasa, a set of simulated kitchen tasks, it went from 66.7% to 68.5%. The gains were not uniform: some individual RoboCasa tasks improved, while others did worse. The overall change is encouraging, but it is not evidence that a robot has acquired a general-purpose understanding of physics.
In fact, the study’s most important caution is built into its design. The model’s learned coordinates and momentum need not correspond to identifiable physical quantities. A mathematically structured hidden space can help predictions behave more consistently without revealing a clean, human-readable map of the world. And the authors note that their tests do not establish robustness to substantially different environments or rollout horizons beyond those examined.
Still, the shift in perspective is striking. A robot’s imagined scene is not just a picture of what might happen. Beneath that picture, a model is carrying a representation forward—one that can either wander with each prediction or follow a more disciplined pattern of change. HWM is an early test of whether giving that invisible layer a little physical structure can make the visible future more dependable. In these experiments, the answer was yes, with limits still firmly in view.
