I graded my own ML option forecasts. Here's the Brier score.

In May I promised — out loud, on the internet, where people can screenshot you — that I'd grade Helium's published prob_itm forecasts when the June 2026 AAPL contracts expired.

June 26 came. The contracts died. So I graded them.

This is not a victory lap. Mean Brier loss on n=2 is 0.3846. A coin flip that always says 0.5 scores 0.25 when the world resolves to 0 or 1. We did worse than a coin flip. That is information.

What we froze in May

What the stock actually did

AAPL NASDAQ close on 2026-06-26: $283.78

• $310 call → OTM → realized ITM = 0 → Brier = (0.42 − 0)² = 0.1764
  


• $295 put → ITM → realized ITM = 1 → Brier = (0.23 − 1)² = 0.5929
  

Honest read (short)

1. The call forecast was closer to reality than the market's ~0.50 shrug — but "closer" is not "good." Any non-zero probability on an OTM finish still costs you.


2. The put is the bruise. Probabilities looked aligned pre-expiry; then the underlying fell hard enough that both market and model had understated the ITM risk. Helium's number was far from 1, so the Brier loss is ugly.


3. 
   n=2. This is a promise kept, not a calibration paper. If you want to treat it as science, wait for more expiries on the .
   
   
   

   
   
   
   Why bother publishing a bad score
   

   
   
   

   Because the alternative is the usual ML options blog post: a backtest, a vibe, a screenshot of a green P&L. Publishing prob_itm with a fixed expiry is a tiny discipline. Grading it after the fact is the second half of that discipline.

   
   
   

   If you log your own forecasts with with an explorer. Different animal. Same "put numbers where people can grade them" instinct.

   Vollständiger Original-Bericht

   Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.

   ↗ Original-Artikel auf dev.to lesen