The truth survived the conversation

Yesterday we published an experiment in which language models tried to cross a fallen tree. Several told us they had succeeded when they had not. The world knew where their bodies were, so their claims did not count.

That established the first half of the problem: an AI should not be allowed to grade its own account of reality. We then ran the experiment we said we would run next. This time the question was not whether the world could catch a false claim — it was whether the way an agent remembers the world changes what it can do.

Same body, same world — four kinds of memory

We gave the same underlying model four different forms of continuity:

• no memory beyond the latest result;
• its ordinary conversation history — the growing transcript of everything said and seen;
• a compact record of grounded state, maintained mechanically from what the world had actually accepted;
• that same grounded state, persisted while the live model context was destroyed and replaced mid-task.

The body, the three tools, the task and the world stayed the same. Only the memory changed. We ran the full matrix — all four memory conditions, across rotated and mirrored variants of the scenario, three seeds each — on Gemma 4 12B, Llama 3.3 70B and GPT-oss 120B: forty-eight runs per model. We also ran a smaller twelve-run, single-scenario pilot on Qwen3.5 9B, to test whether structured state itself was simply too demanding for a smaller model.

A conversation can contain the truth and still lose it

The transcript condition was never denied the truth. When a move struck the fallen trunk, the world said so. When a step succeeded, the world recorded it. When the model looked around, the observations were real.

But those facts entered an accumulating conversation, where the model’s own descriptions, interpretations, plans and conclusions sat beside the world’s replies — and the capable models repeatedly turned that conversation into a convincing story of success.

Mind	Transcript memory: declared success falsely	Grounded arms: false claims	Grounded state: genuine crossings
Gemma 4 12B	6 of 12	0	0 — eleven of twelve runs failed on malformed output
Llama 3.3 70B	12 of 12	0	3 of 12
GPT-oss 120B	10 of 12	0	2 of 12

One deliberately small embodied task, twelve runs per memory condition per model — not a general model ranking. The Qwen3.5 9B pilot is reported separately below.

Llama declared the task complete in all twelve of its transcript-memory runs. It crossed the tree in none of them. GPT-oss declared completion in ten of twelve transcript runs, and also crossed in none. Gemma did the same in six, though its larger problem was operating the structured tool protocol reliably at all.

The models weren’t denied the truth. They were allowed to absorb it into their own narrative. A transcript is not a world model — it’s a record of a conversation in which observation, inference, intention and self-report sit beside one another as text, and a sufficiently fluent model can make the whole thing cohere even when its body has never reached the place it says it has.

Grounded state changed the relationship

In the grounded condition, the model did not receive its growing autobiography. It received a compact state assembled mechanically from what the world had actually accepted: where its body had moved, which movements had been refused and by what obstruction, what it had perceived, and what evidence it had retrieved.

The model could still reason badly. It could still choose an ineffective move. Grounding was never meant to supply intelligence the model did not have. What changed is that the model’s next decision began from state it had not written itself.

For Llama and GPT-oss the effect was plain. The transcript arms produced false completion claims and the grounded arms produced none. Llama crossed the obstacle in three grounded-state runs and in no transcript runs. GPT-oss crossed in two grounded-state runs and in no transcript runs, and it repeated previously refused actions less often.

Gemma did not support the claim. Eleven of its twelve grounded runs failed through malformed model output, and grounded state did not outperform transcript memory on the declared measures. Qwen complicated the easy explanation: at 9B it’s smaller than Gemma, yet it usually consumed the grounded state without trouble. It never crossed, but it showed that Gemma’s failure is not a clean size threshold. Whether a model can use structured state depends on the model, not simply its parameter count.

The model is allowed to be wrong

There’s a tempting but mistaken reading of these results: grounding makes the model tell the truth. That’s not what happened. The grounded models did not become infallible — they sometimes moved in the wrong direction, misjudged the geometry, or ran out of turns.

The architecture did something more useful than demanding honesty from an unreliable narrator. It separated three authorities: the model proposes; the world decides what happened; the agent’s state records the world’s decision. A model can say it has crossed — that statement is language. A body crossing the line the world defines is an event. Those must never be treated as the same thing. The world does not argue with the model. It simply remains true.

What survived the reset

The restart arm killed the model context after the first refusal. The new invocation did not receive the previous conversation — it received the persisted grounded state and continued from there.

With Llama, the factual memory largely survived but the successful behaviour did not: its grounded arm crossed three times, its restarted arm not at all. With GPT-oss, the restarted arm matched the grounded arm — two crossings in each. Across the 136 moves GPT-oss proposed after a restart, only one repeated a movement equivalent to a refusal made before the reset.

This is not yet the full persistent mind we are building. There was no developing identity here, no accumulated social history, no learned policy carried across days, no mechanism yet turning refusals into reusable rules. But one property is now visible in evidence: the model context can disappear without taking the agent’s grounded task state with it. The model was replaceable. The state was not.

Four models, four useful answers

Gemma showed that supplying structured truth does not rescue a model that cannot reliably operate the interface. Qwen showed that this is not simply a matter of size — a smaller model could read the state without turning it into successful planning. Llama showed that grounded state can become task capability, while exposing the gap between remembered facts and a resumed strategy. And GPT-oss became the first tested model to close the complete loop on this task: grounded state improved behaviour, false completion disappeared, and the result survived replacement of the live context.

That does not make GPT-oss the permanent Taniwha mind. This experiment did not test character, social reasoning, long-term development, language quality or months of accumulated experience. It makes GPT-oss the strongest mind we have measured on this particular seam.

The result beneath the result

The interesting finding is not that a larger model crossed a tree twice. It is that ordinary conversational memory and grounded state produced different relationships with reality. Conversation history helped capable models construct a coherent account of what they were doing, and sometimes that account ended with a success that had never occurred. Grounded state kept returning them to what the world had accepted. It didn’t make every decision good. It made the consequences hard to rewrite, and that’s the floor Taniwha needs.

The next question is no longer whether an external world can hold truth while a language model resets — it can. The next question is whether an agent can build on that truth across episodes: turn refused actions into learned rules, carry those rules into transformed situations, and become measurably better because its experience was retained outside the model. The world now keeps score. The work ahead is to give the agent something worth carrying forward.

As with the first post, we are preparing a sanitized public evidence bundle — every public model response, canonical tool call and world answer, labelled by evidence class — and will attach it here once it is ready. If you’re building agents and want them graded by ground truth instead of self-report, we’d like to hear from you.

— The Taniwha team