Five Fine-Tunes That Never Learned to Think

I am not a data scientist. Until last week I had never built a training dataset, never read a paper about building one, and had a working theory that I did not need to: dump a knowledge base into the training data, fine-tune, and see what the model can do. The interesting engineering was in the harness anyway. That is where I had spent months.

The model I was fine-tuning is Emil, an LLM-driven NPC that lives on my Minecraft server — Nemotron 3 Nano 30B-A3B in a Rust agent loop, running locally on a DGX Spark. Every training cycle is LoRA, merge the adapter into the base, quantize to NVFP4, serve, and run a five-stage capability bench: collect three logs, craft a wooden pickaxe, craft a stone pickaxe, build a furnace, make torches. Five rungs of the earliest game.

Over nine days I trained three checkpoints on data I had never opened. This is what that cost, and what I built once I finally read it.

The overnight run

The first dataset started with a research pass I ran locally. DeepSeek-V4-Flash — 284B total parameters, 13B active, 1M context — served on DwarfStar, a native inference engine for that model family that runs on the Spark. I pointed it at the question of where Minecraft training data comes from and let it run for a night.

It came back with a good answer: existing datasets worth pulling (minecraft-question-answer-700k, Voyager’s skill library, the CraftAssist dialogue corpus, Andy-4), their licenses, and the sanctioned bulk-access paths for the wiki and Arqade. Claude turned that list into an assembled dataset.

I did not open it. That decision is the whole article. My reasoning at the time was that a small knowledge dataset cannot hurt much, so the mix was not worth my attention, and the harness was.

Three checkpoints, three theories

npc-v6 trained, merged, quantized, served, and was degenerate. Identical remember and goto calls repeating inside a single turn, equipment it never had, knowledge-corpus text regurgitated verbatim into memory notes. The postmortem blamed the mix: 58% synthetic knowledge QA, two epochs at 2e-4, classic overfit. Reasonable.

npc-v7 was the fix for that. Clean start: I quarantined 39 poisoned episodes, purged everything recorded against an older, dishonest version of the harness, and trained episode-free by design on curated scenarios. It was bad enough that I later made it a rule never to use it as a baseline for anything.

npc-v8 was the fix for that. If curation is the problem, curate properly: twelve Haiku agents reviewing all 3,309 examples one at a time, verdicts stored, applied at build. It found real things — an entire “chemistry” contamination bucket that had come in from Education Edition wiki pages, 33 scenarios that crafted a furnace’s output instead of smelting it.

Then I benched all of them against the untuned base model on the same seed, same suite, same bot revision:

weights	stages passed	tool calls	tool-error rate
npc-v6	0/5	851	84.3%
npc-v8	0/5	686	94.9%
untuned base	1/5	366	45.1%

Every fine-tune was worse than the model I started from, and the harder I cleaned, the worse it got. All three failed the same way: the model could not let something in front of it override what it already believed about Minecraft. The harness would tell it, in the observation, that craft{item:WoodenPickaxe} works right now — and it would ignore that for dozens of turns while acting on a world it had imagined and writing memory notes about progress that had not happened.

The machinery I built instead of looking

By this point I wanted to know which data was doing the damage, so I built lineage. Every dataset build now writes a manifest with a content-derived dataset_id; the training script stamps it into the run directory and refuses to start if it does not match the actual training file; merge and quantize carry it forward into the model directories; the bench records it in each run’s metadata. One script joins the whole chain, so any bench result can name the exact data mix behind the weights that produced it.

I still think that was worth building. It could not have found this bug.

Neither could the curation pass, and that one hurts more, because twelve agents spent a long time reviewing text that was never the problem.

Reading what the tokenizer actually emits

Four sessions of Claude had been working across this system on different optimizations, and every one of them pointed at the harness. That is a reasonable prior: the harness is entirely custom, thousands of lines of my Rust, and the base model’s chat template is the one component nobody wrote and everybody trusts.

The template is a Jinja file that ships with the model. It renders a list of messages into the exact token string the model trains on and serves on — the same function on both sides. Mine contains this:

{%- if '<think>' not in content and '</think>' not in content -%}
    {%- set content = "<think></think>" ~ content -%}

Any assistant message with no reasoning attached gets an empty think block prepended. A few lines further down, a flag called truncate_history_thinking — default True — strips the reasoning out of every prior assistant turn and replaces it with the same empty marker.

Both behaviours are correct for inference. You do not want to re-feed stale reasoning into a model’s context on every turn; NVIDIA is doing the right thing. They are fatal for training, because at training time those same rewrites turn every assistant message into a target that says: given this situation, think nothing, then act.

I checked how many rows in my data carried a reasoning field. Zero. All of them. Every row I had built, curated, quarantined, and traced through five stages of lineage had been converted into an empty-think target at tokenization, which is downstream of every tool I had built to inspect it. The JSONL was fine. The manifest was fine. The rendered string was the problem, and nothing I owned ever looked at the rendered string.

It got worse on the way out. Training on empty thinks also broke the model’s ability to close a think block at all, which forced NO_THINK at serve time to get parseable tool calls back. That silently disabled a feature I had shipped days earlier and believed I was running: two-speed cognition, where planning turns think properly and acting turns do not. The bot sent thinking: None on plan turns, which inherits the server default, which was off. For three days the slow lane did not exist and I did not know.

Two amputations, one wound

Once I could see it, the dataset content stopped looking innocent too.

The current build is 3,559 training examples, and 1,709 of them — 48% — are knowledge QA: a system prompt, a question, an answer. Nothing in between. I had chosen a format that teaches the model to map a question straight to an answer with no reasoning, and then the template had stripped the reasoning out of everything else. The same mistake at two layers, one deliberate and one invisible.

Nemotron is a reasoning model. Instruction-following and the ability to let evidence override a prior live in the think phase. I had spent a week removing the think phase and then measuring how badly the model reasoned.

Making thinking-native data

The rebuild is a pipeline with a gate at every stage where I got burned.

The dataset builder now attaches a think to every assistant message and refuses to emit the file otherwise — 6,871 of 6,871 messages covered, verified by rendering the output back through the tokenizer and counting empty think blocks. The training script raises on any <think></think> in a rendered training string, which is the last line of defence at the exact layer where the damage hid.

Then there is the question of what to put in those thinks.

About 13% of my episode turns logged the model’s own reasoning at play time (241 of 1,866). Those get distilled: the local server compresses each turn’s real reasoning down to one or two first-person sentences, so the model learns to think briefly and act rather than deliberate for 500 tokens.

The other 87% had nothing, and the tempting fix is to show a generator the action that was taken and ask it to write the reasoning. That is motivated reasoning, and it manufactures a confident rationale for whatever it is handed, including the flail turns my logs are full of. It is the same disease as before: text that looks like thinking, in the slot where thinking goes.

So the synthesizer never sees the action. It gets the observation and the retrieved memory from before the decision, runs the base model forward, and lets it decide for itself. The reasoning is kept only if the action it independently chose matches the action actually logged. Disagreement is a free deterministic discard of bad turns — no error flags leaking in, no LLM judging its own work.

It accepts about 18% of the turns it is offered. That number is the point, not a caveat. A wall that let 90% through would be rationalizing, which is the failure it exists to prevent, and anything it rejects still gets a grounded one-line fallback, so no row is ever empty. The literature agrees on the direction and is specific about the cost: synthesis with the answer exposed scored 0.47 below the base model, while hidden-answer forward synthesis scored +3.79 (Context-CoT, arXiv 2605.25354); STaR keeps only rationales that reach the target and throws away the rest.

The first dataset out of that pipeline has zero empty think blocks anywhere in its rendered output. Every assistant message carries a think, the trainer refuses to run if any of them render empty, and the reasoning that is in there was either the model’s own or independently derived without ever being shown the answer.

Before trusting a dataset now, I render one example through the tokenizer the way training will, and read the string the model is actually going to see. It costs a minute. I got to it sixth, after sourcing, generation, quarantine, lineage, and a twelve-agent review, all of which were inspecting a file that was never wrong.

Postscript: npc-v9 and npc-v10. The first checkpoint off that pipeline cost about nine hours of training and taught me nothing I did not already know.

I had handed the synthesis run to an agent. It processed 10 of the 26 episodes in the pool and handed the result back as the finished job. The one number that decided whether the run was worth training on — how much of the pool it had actually covered — never made it into the report. A partial sidecar is the same kind of file as a complete one, so the build picked it up without complaint, and 171 of 3,559 examples ended up carrying real reasoning. The rest fell to the fallback, one sentence composed from the action the turn took. The model learned that faithfully: every think block it produced on the bench read like Next: mine_block.

I knew exactly what I was fixing when I started that cycle. I lost the day to a fraction of a job described as a whole one. The build does have a gate here — it aborts when the number of attached thinks is zero, which is the number it was written for. 171 is not zero.

The next afternoon I rebuilt the dataset and trained npc-v10, which failed the same way. I only understood why this week, putting the two dataset manifests side by side.

They are the same file. Same builder commit, same source mix down to the row, same curation verdicts, same LoRA rank and targets. One field differs: real thinks attached, 171 against 188. Seventeen rows out of 3,559.

npc-v10 was not a fifth theory. It was the fourth one retrained without fixing what broke it, because rebuilding is one command and re-running the synthesis is nine hours, and nothing in the build refused. The lineage chain I built instead of looking never did catch a bug. It just let me prove, a week late, that two failures were one.