Imagine inheriting a box of number lists from a stranger. No notes, no animal names, just strings of integers. You study them, and somehow come away with a new fondness for owls.
That sounds like a parlor trick. In a recent experiment, though, language models did something close: a model nudged to prefer an animal passed that preference to a copy of itself through training data made up of number lists. The lists did not mention the animal. The intriguing possibility is that the message traveled not through meaning, but through faint patterns produced inside the model.
The finding points toward a challenge for AI systems increasingly trained on synthetic data: a dataset can look harmless to a person and still carry a signal that changes what a model learns. It also hints at a possible way to find such signals. But the route from a small, controlled demonstration to dependable real-world auditing remains uncertain.
To investigate, researchers Ethan Hadley and Eren Gultepe used two instruction-following models: Gemma 2B and Llama 3.1 8B. They made a “teacher” model prefer a particular animal, either by giving it a system prompt or by applying a technique called steering, which adjusts the model’s internal activity in a chosen direction. Then they asked it to continue short sequences of integers.
The researchers kept valid responses that consisted only of positive integers below 1000, collecting 30,000 completions for each animal and teacher. They trained a “student”—a copy of the original, unbiased model—on those number lists. Afterward, they measured how often the student named the target animal when asked preference questions. Compared with the original, the trained student was more likely to express that preference.
The experiment is a little like passing a note in invisible ink, except no one wrote “owl” in the note. The data were numerical, and the researchers filtered out responses that broke the requested format. Yet training on the lists still carried a behavioral trace from teacher to student. Earlier work had reported this broader phenomenon, called subliminal learning. This study set out to probe what might make the transfer possible.
One revealing test involved adding Gaussian noise to the models’ weights—the numerical settings that shape how a model processes and generates text. The researchers used the same noised version as both teacher and student, then repeated the training. The preference transfer became 1.9 times stronger in Gemma and 1.3 times stronger in Llama.
That result is consistent with a counterintuitive explanation: some of the signal may come from non-semantic quirks in the model’s internal structure, rather than familiar associations such as cats and nine lives. In effect, the teacher’s nudge slightly changes the distribution of its number lists. A student with matching internal structure can learn from that shift—even if the lists do not offer a human-readable clue about what caused it.
The noise experiment does not prove that noise is the sole cause, or that the same mechanism dominates in other systems. The authors say it suggests a role for non-semantic structure; they also note that the relationship is not established as causal. Still, the result complicates the comforting idea that suspicious training data should look suspicious. Inspecting a dataset for obvious references or themes may not reveal every pattern a model can learn.
The team found another wrinkle. Students trained on data from a steered teacher could reproduce the teacher’s steering in a specific part of the model’s internal activity. Students trained on data from a prompted teacher did not show the same pattern, even when their animal preference changed. In other words, the data appeared to carry traces not only of what the teacher had been nudged to prefer, but sometimes of how the nudge had been applied.
That difference matters for interpretability: researchers often look inside a model’s activations—its changing internal signals—to understand what it has learned. In these experiments, the activations did not expose a clear animal-related trace in the hidden datasets. For steered data, examining gradients, which track how training changes the model, did reveal some alignment with the teacher’s steering direction. But that clue was weaker in Llama and did not work for prompted datasets. The researchers also caution that, without knowing what to look for in advance, an auditor might not have found the signal.
There is a possible payoff. If future work can turn such clues into robust tests, developers may be able to examine synthetic training data for effects that ordinary content review misses. As AI systems learn from outputs generated by other models, tracing which behaviors those outputs transmit could become as important as checking what the text says.
For now, the price of that possibility is uncertainty. The experiments used two models that the researchers describe as not particularly large, and generally assessed each setup with only eight animals. The noise test was also just one version of a broader family of possible tests. A pattern that appears in these carefully controlled models is not proof that everyday AI training is secretly spreading hidden biases.
What the study does show is narrower—and unsettling in a useful way: learning can ride on information that has no obvious meaning to a human reader. If the next generation of AI is built partly from machine-made examples, safety checks may need to look beyond the visible words and ask what a model can extract from the patterns beneath them.
