800 Million Tokens for $36: Benchmarking DSH + DeepSeek V4 Pro on a Real Codebase

There is a problem with most benchmarks for coding agents: they are not software development. They are useful, of course. Give an agent an issue, run a test suite, check whether the patch passes. SWE-bench and similar evaluations give us a standardized way of comparing models. But this is not how I actually use agents. I don't want an agent to fix...