An AI agent can fail to reach a destination even when its training data contains all the pieces of a successful route. It may have seen how to travel from one area to a familiar waypoint, and separately how to continue from that waypoint toward the destination. But if it never practiced that particular start-to-finish journey, a policy trained to imitate examples may not join the pieces together.
This gap matters in reinforcement learning, where an agent learns by relating actions to the outcomes they produce. A common shortcut is to treat the task as supervised learning: show the model a state, a desired goal, and an action that previously helped reach that goal. It is a straightforward recipe, but it can reward memorizing familiar state-and-goal pairings rather than learning how to combine them in new ways.
Researchers Raj Ghugare and colleagues gave that problem a precise name: combinatorial generalization. An agent may recognize the starting state and the goal individually, yet have never encountered them together. The challenge is not simply recognizing a new place. It is assembling known pieces of experience into a route the training data did not record as one continuous trip.
Testing that ability requires care. The team found that popular maze benchmarks could make the challenge look easier than it was: many supposedly unseen start-and-goal combinations already appeared in the training data. So they built new tests in which an agent had to navigate between regions that had been visited separately. The test asked a sharper question: could a learner recombine what it had experienced, rather than succeed by replaying a familiar pairing?
Their proposed remedy is a small change to the training examples, called temporal data augmentation. Instead of changing an image or cropping a signal, the method changes which future goal is attached to an example. First, it groups nearby states into clusters using a distance measure. When a training example has a goal in one cluster, the method chooses a nearby waypoint from that cluster, then selects a later state from a different recorded trajectory that passed through the waypoint.
The original action is now associated with a goal beyond the waypoint, even though that full journey was not recorded in one trajectory. The intuition is that if the original goal and the waypoint are close, an action that starts moving toward the original goal may also be useful for reaching the waypoint. From there, a different trajectory supplies the onward destination. Repeating this during training teaches the policy about combinations that were missing from the original examples.
This is not a route planner that searches through saved paths at the moment it is asked to act. It is a way of enriching the lessons used to train a policy. In the researchers’ experiments, the approach could be added to existing outcome-conditioned learning methods, including a simple neural policy and a transformer-based Decision Transformer. The key ingredient is temporal: it connects one recorded episode to another through a nearby point in the sequence of states.
In point-maze and ant-maze tests, adding the augmentation improved goal-reaching performance for the simple policy on five of six tasks. The experiments also included image-based environments. There, the researchers found that even a basic distance measure could help on some tasks, although they cautioned that finding a useful measure becomes difficult when states are high-dimensional. The method therefore offers a practical idea, not a universal fix: its joins depend on recognizing which states are genuinely close or interchangeable.
The team also checked whether more training data or a larger model could substitute for the new examples. In their Decision Transformer tests, multiplying the dataset size by ten and increasing model depth did not substantially improve the specific kind of generalization they were measuring. In those experiments, simply showing more of the same pairings did not reliably teach the model to combine them.
A later study by Michał Bortkiewicz and colleagues complicated that conclusion. Rather than add temporal examples, they tested different reinforcement-learning methods in a carefully controlled grid where an agent picked up and placed boxes. Training might require moving boxes between neighboring sections; evaluation could ask for a transfer between diagonal sections. In another setup, agents learned tasks involving fewer boxes and were tested on tasks involving more.
That study found that temporal-difference learning, often thought to be especially suited to combining experience, had an advantage in some exact stitching tests—but was not enough by itself as tasks grew harder. Monte Carlo methods could also combine behaviors in some settings, particularly when test states stayed within the range of states seen during training. Increasing the capacity of the value network helped both kinds of methods narrow the gap between familiar and unfamiliar tasks.
The two approaches are not interchangeable. Ghugare’s team deliberately augmented offline examples so supervised-style policies could practice missing combinations. Bortkiewicz’s team studied online agents learning in a controlled environment and found that network capacity affected their ability to generalize. Together, the studies suggest there is no single switch labeled “stitching.” What helps depends on how data is collected, whether a new route passes through familiar states, and how much the model can represent.
That last distinction is crucial. A route assembled from familiar segments may still lead through an unfamiliar situation, where the agent has no reliable experience to guide it. The newer study’s grid was intentionally simple so the researchers could isolate the stitching problem; its results do not establish that scaling alone solves harder, real-world planning. Likewise, temporal augmentation relies on a meaningful distance between states, which may be hard to define when an agent sees images or complicated physical situations.
Still, the underlying lesson reaches beyond mazes. When an AI system seems to know each ingredient but fails at a new combination, the missing ability may not be more memorization. Researchers can instead ask whether training has taught the system how familiar events fit together—and build tests that distinguish genuine recombination from replay. Sometimes, the route to a new capability begins by changing not what the model sees, but how its existing experiences are connected.
