My NPC Tried to Place a Furnace It Didn't Have — 69 Times

My freshly fine-tuned Minecraft agent (Emil — an LLM-driven NPC, introduced properly in that post) came out of training with its best eval loss ever. Then it went into the behavioral eval — thirty minutes alone in a fixed-seed world — and came back with 1,450 tool calls, 856 of them errors. All 98 craft attempts failed. And the highlight: it tried to place a furnace 69 times while owning zero furnaces, getting “you don’t have any Furnace — get some first” after every single attempt.

Loss said the model was great. Behavior said it was broken.

I debug this project with a coding agent in the loop — it babysits the runs, greps the logs, writes the throwaway analysis scripts. What follows was a dialogue between its legwork and my questions.

“What do you mean, perseveration?”

The agent’s first summary called this perseveration: repeating a failing action instead of adapting. A label is not a mechanism, so I made it unpack the errors. Most were my own guardrails firing — a repeat-chat filter, a call-breaker that refuses a call after three identical failures. The guards worked. The model just kept feeding them.

But why would a model trained on episodes from a system with feedback learn to ignore feedback?

My hypothesis: the flywheel poisoned itself

Emil’s training data comes from earlier runs of Emil. If those episodes contained retry loops, the fine-tune would have learned that after an error, you repeat the call. So: analysis script. Result: 100% of post-error calls in the training data were identical repeats.

Except the number was garbage. The script assumed OpenAI-style tool-call JSON; my dataset stores the fields differently, so every call parsed as empty and compared equal to everything. Rerun with the schema read correctly: clean data. After an error, the recorded behavior switched to a different call 47 times out of 47.

When an analysis agrees with you that emphatically — 100%, first try — audit the analysis before you celebrate. An agent will hand you a broken number with a straight face. Verifying it is the job.

The timestamps

The agent dug into when the 69 placements happened: gaps of median 2,045 ms, minimum 2,030 ms. Metronomic. A model that reads an error and retries doesn’t tick like a clock; a dispatch queue draining at a fixed rate does.

That cracked it. My harness streams tool calls out of a generation and executes them at ~2 s each, buffering results. The model didn’t ignore 69 error messages — it wrote all 69 calls in one generation, before the first error existed. 1,450 calls over 27 turns is ~54 calls per generation, from a model whose training data averages one call per turn. A train/serve mismatch, hiding in a 2,030 millisecond gap.

My call: the speed stays

The obvious fix was turning the streaming dispatch off. I pushed back — dispatch is why Emil acts fast, and in a world with hostile mobs, speed is survival. Instead we ablated it: same model, same seed, dispatch off.

Same behavior. Same ~50-call floods. The dispatcher only changes when queued calls execute, not how many the model writes. The plumbing was innocent, the weights never learned to stop — so I keep my latency edge, cap calls per generation in the harness now, and teach multi-call turns properly in the next training cycle.

What I’m keeping

The checkpoint didn’t ship, and the investigation was worth more than the checkpoint. Behavioral evals catch what loss can’t. Timestamps are evidence. And an agent in the loop multiplies your throughput exactly as much as your questions and your verification deserve — it found the metronome, and it also confidently handed me a fake 100%. Both are what working with agents is actually like.