I Rejected a Model at 44 tok/s. The Model Was Doing 108.
- Published
- Author
Swapping the base model under my Minecraft NPC: three configuration mistakes worth 2.5x, a container that crash-looped 23 GiB nine times, and a context-window constant denominated in a unit that changes size when the model does.
Read more