I Raised Gemma 4's Token Cap. The Dense Model Stopped Refusing.

TL;DR: Last week I argued Gemma 4 Dense regressed on a grounded-retrieval scenario under tightened prompts — and called the MoE-vs-Dense divergence architecture-mediated. The comment thread, led by Robin Converse on her sovereign Ollama stack, proposed an alternative: my max_tokens: 400 cap was starving Gemma's reasoning layer before the visible reply completed. I re-ran the same six scenarios with one variable changed — budget raised from 400 to 4096. Dense recovered on every scenario, including the false-refusal headline that anchored the original article. MoE did too. The original MoE-vs-Dense divergence largely disappears when reasoning has room to finish. The cap was doing the work. Walking it back publicly.

Why I Re-Ran It

Last week I published ( and asked: what does the same test look like on the managed Gemini API side? She also separately filed two upstream Ollama bugs — walked back her framing on was confirmed by multiple users and resolved in a later release. That kind of fair-witness practice is what made me trust the hypothesis enough to test it.

The hypothesis itself, sharp: my max_tokens: 400 cap was starving Gemma 4's reasoning layer before the visible reply completed. Capability ceiling and orchestration pressure look identical from the outside, as mapped a separate substitution-vs-decision boundary that sharpened where to look. , each named structural reframes I had to engage. The collective shape of the thread was: the cap is one variable, the architecture is a category, and you've been calling the result by the wrong name.

The cap was the part I could test fastest. One variable. Re-run the same six scenarios on the same two architectures with the only change being the budget.

The Experiment

Single-variable change against the original v2 conditions:

The implementation was four lines in the chat-models server file called it from her stack on Tuesday. The cross-validation now exists: same architecture, two deployment contexts (sovereign Ollama, managed Gemini API), one finding — uncap the budget and the failure mode evaporates.

What still partially holds from the original article:

• 
  Reluctance, not hallucination, as the dominant failure mode for grounded Arabic chat on open models when the budget is too tight to complete reasoning.


• 
  Variant-specific prompt tuning is real — but the variant-specific thing isn't an architectural slot; it's a token budget shaped to the variant's reasoning footprint.


• 
  Latency on the Google API is still a chasm for interactive chat — that part wasn't a cap artifact.

What I retract:

• The "architecture has slots for sequential sub-behavior that dense doesn't" hypothesis was too strong. Dense plus budget gets the same grounded behavior.


• The matrix's "31B Dense regression" reads now as "31B Dense under-budgeted regression."


• The closing line — "I think I was tuning architecture, not size" — should have been "I think I was tuning budget, mediated by architecture."
  

What's Still Open

• 
  Temperature dimension untested in this run. I held temperature at 0.3 to keep this one-variable. proposed making preconditions first-class through his NEXUS protocol; shared that JAMES's per-stage DEFAULT_MAX_TOKENS defaults (200/400/400/400 across four cognitive stages) match this pathology — their 2026-05-18 internal eval reported empty responses on gemma4:e4b at exactly those four stages. Uncapped replication is queued this week. If it reproduces, that's three independent deployment contexts (sovereign Ollama, managed Gemini API, JAMES production) on the same cap pathology before any cross-experiment swap runs.

The Lesson

The community ran my falsifier for me. The cap is one variable; architecture is a category. If your model is failing under a tight token budget, raise the budget before you reach for architectural explanations — and if a thoughtful commenter offers you a single-variable test against your strongest claim, run it.

The article you can write a week later, with the comment thread folded in, is stronger than the one you ship alone. Robin Converse is the right person to share the next round with.