Someone offers a deal: If you finish the garden, I’ll pay for dinner. The garden isn’t finished. Did they pay? Strict formal logic says the condition doesn’t tell us whether dinner was paid for some other reason. But in everyday conversation, most listeners hear a more specific promise: finish the garden, and you get dinner; don’t finish it, and you don’t. Now consider: If you’re hungry, there’s bread on the counter. You aren’t hungry. The bread is still there. Same little word, different inference. The contrast points to an invisible layer of conversation: what we assume a speaker means beyond what the sentence literally says.
That layer is not a decorative extra. It helps us work out whether a promise is conditional, whether information is simply being offered, and what a speaker expects us to understand. A computer can follow the formal logic of an if-statement and still miss that social calculation. To see how often that happens, researchers compared people and language models on carefully controlled sentences in four languages.
The study tested 313 native speakers of Catalan, English, Italian, and Spanish, alongside 25 language models. Participants read short contexts and conditional statements, then judged whether the stated outcome was true. Some cases were straightforward controls. Others were designed to make literal logic and ordinary conversational inference pull in different directions: the condition had not been met, but the listener still had to decide whether the outcome followed.
People made the distinction in ways the researchers could measure. For standard promises, participants inferred an “only if” meaning more than 88% of the time on average across the four languages. For statements like the bread-on-the-counter example—often called biscuit conditionals in linguistics—they inferred that the outcome held independently of the condition 57% of the time. That second result is less decisive: context-sensitive understanding is not a single reflex, even for people. But human responses did shift with the kind of statement.
The models’ responses shifted less reliably. In the critical cases, their accuracy fell below the human level, and the researchers found that models rarely made the context-independent inference. Individual results varied sharply: average accuracy in the experimental trials ranged from 0.000 to 0.833. That spread matters. There wasn’t one uniform model weakness so much as a collection of different habits hiding behind fluent answers.
One habit was to stick closely to the formal logic of if: when the condition was false, treat the conditional as true, whatever the surrounding context suggested. Another was almost the opposite—a blanket shortcut that treated every conditional as if it meant “if and only if.” That second rule can work for a promise, where the speaker implies the condition is necessary, but it misfires when the sentence simply gives information that remains true either way. Neither habit flexibly tracks why someone chose that particular sentence.
A high score could conceal the same problem. In the researchers’ test, Llama 3.3 scored 0.972 on the critical standard cases but 0.167 on the biscuit cases. It handled one conversational pattern far better than the other. The result is a warning against treating a model’s overall accuracy—or a convincing explanation—as proof that it is interpreting context as a person would. A system can land on the right answer through a rigid rule, then fail when the conversational situation changes.
The researchers also looked for broad features that might explain which models did better: whether they were open or closed, built with dense or mixture-of-experts architectures, or trained as general-purpose or reasoning-oriented systems. None reliably predicted performance on this task. Models with similar labels could behave quite differently. The source of the differences remains unclear; the study does not identify a specific internal mechanism that produces human-like inference.
Language itself offered another clue. Across these four languages, the broad pattern of results was similar. But some models answered prompts in English even when the prompt was in Catalan or another language. The researchers note that this could point to English’s stronger role in some systems, while cautioning that a wider language sample would be needed to judge how general the pattern is. A model may appear multilingual yet still lean on a narrower linguistic center of gravity.
The authors describe the main behavior as a decontextualization bias: a tendency to favor literal wording over contextually enriched interpretations. That label summarizes a pattern in the answers; it does not prove what is happening inside a model. The experiment measured judgments about short, controlled sentences, not conversation unfolding over time. Its four languages are also related, and richer dialogue might provide cues that change how people or models respond.
Still, the test makes a subtle distinction visible. Understanding a sentence can mean recognizing its formal structure; understanding a speaker can require inferring what their choice of words implies in this situation. Those abilities overlap, but they are not identical. The little word if opened a narrow window onto the difference—and showed that a fluent answer can leave the most human part of the message untouched.
