A reader named Ryan left the sharpest comment on my last post. The gist: it was jargon-heavy, kept restating "you have to test the AI's output" in new words, and read more like a pitch deck than a case study. He was right, and he pointed at the fix himself: walk through one real task. What changed, what caught the problem, what proof was required, where did a human step in.
So here is one task, start to finish. Everything below is public and runnable. No withheld details, no diagrams of boxes with arrows.
The setup, in two sentences
My match-3 game runs on a plain-Java rules engine. It runs two ways: on a JVM (where the tests and CI live) and in the browser, compiled to JavaScript by (BoardEngine.java, IdUniquenessTest.java).
The same trick, one level up
Around the same time, the same "run it two ways, look for a quiet disagreement" habit pointed the other direction. Auditing where TeaVM and the JVM diverge, I hit a date case they split on:
YEAR = 2002, WEEK_OF_MONTH = 2 (America/New_York, en_US)
JVM: Sun Jan 06 2002
TeaVM: Sat Jan 12 2002
This time my code was fine. The bug was in TeaVM's reimplementation of Java's GregorianCalendar. One line used days - 2 where every neighbouring branch, and the Apache Harmony code it was ported from, used days - 3:
-days += (fields[WEEK_OF_MONTH] - 1) * 7 + mod7(skew + dayOfWeek - (days - 2)) - skew;
+days += (fields[WEEK_OF_MONTH] - 1) * 7 + mod7(skew + dayOfWeek - (days - 3)) - skew;
One character. The reason no test had caught it: the suite already had four assertions that reach that exact line, commented out since 2015. My change re-enabled them instead of adding new ones. They fail on the old code and pass on the fix, which is the cleanest proof I could ask for that the fix is real and nothing else moved.
It — the Java engine, 59 tests, playable in-browser via TeaVM.
evals-differential-oracle — a tiny browser demo of the same idea: the same match-3 rule written twice, fuzzed against each other over thousands of boards, plus a deliberately-broken version both nets catch.Thanks to Ryan for the nudge. The last post told you I test things. This one showed you one.
SOCIAL SHARE CARD GENERATOR