Most model comparisons ask which model is best. This one starts with a model that never even produced a single result.
We tested NVIDIA's open-weight Nemotron family, from the 30B Nano to the 120B Super, on a benchmark of real-world coding tasks: the kind of models an indie developer on a tight budget, or an enterprise cutting inference cost and keeping data in-house, would run.
The main finding is that model size is not a dial you turn for a little more quality, it is a threshold. Below a certain capability floor a model cannot drive an agent loop at all, which is why the smallest variant we tried, Nano 12B, produced nothing to score.
Above the floor, the question stops being which model is cheapest and becomes which one clears the bar your work actually needs: Nano 30B is an extremely cheap workhorse for narrow, well-scoped jobs, while Super 120B is the size that holds up on demanding multi-step agent work.
An agent size floor is the minimum model capacity below which a model cannot reliably complete the act-observe-decide loop an agent depends on. Below it you don't get a slower or sloppier agent, you get a non-agent: a model that reads the task, takes a few steps, and never converges. For anyone choosing a model, this changes the question from "which is cheaper" to "which clears the floor for my work", and that is the question to answer first.
Where the numbers come from
Every scenario in the evaluation is a real-world agent task tied to a published skill, scored on two axes: instruction-following (does the agent do what it was told, in the way it was told) and task-completion (does it reach the goal). The overall score weights instruction-following at 4 and task-completion at 3, then divides by 7. Each task runs with and without the skill, so the lift from the skill is visible directly. The tasks and skills are public, in the is where you find the skills that take it the rest of the way.
SOCIAL SHARE CARD GENERATOR