If you follow AI coding tools, you have probably seen companies quoting their SWE-bench scores in every product announcement and marketing page. But what do these numbers actually mean? And more importantly, should you pick your AI coding tools based on benchmark scores alone?

SWE-bench has become the de facto standard for measuring how well AI...