Smash Stories: The Bug That Whispered for Two Weeks Before I Heard It

python

pattern = r'[₦$£€]?\s?\d{1,3}(?:,\d{3})*(?:\.\d{2})?'

matches = re.findall(pattern, extracted_text)

if not matches:

    return False, "", "No currency values found"



This alone catches about half the errors garbled text, partial reads, and non-numeric data that OCR sometimes returns.



Layer 2: Double-Read Confirmation



The agent captures two separate screenshots and runs OCR on both independently. If the numbers match, the value is accepted with high confidence. If they differ, a third screenshot is captured and the best-of-three wins.



# Read twice

text1 = extract_text(screenshot1)

text2 = extract_text(screenshot2)



num1 = extract_number(text1)

num2 = extract_number(text2)



if num1 == num2:

    return True, num1, "High confidence"



# Tiebreaker

text3 = extract_text(screenshot3)

num3 = extract_number(text3)



if num1 == num3:

    return True, num1, "Medium confidence (2/3)"

elif num2 == num3:

    return True, num2, "Medium confidence (2/3)"



This catches transient OCR errors—pixel-level variations between screenshots that produce different readings. The cost is 4-6 extra seconds per financial read. The benefit is catching errors that would otherwise silently corrupt data.



Layer 3: Range Validation



Even a correctly formatted, double-confirmed number can be nonsensical. A balance of "₦0" might mean an empty account—or it might mean the OCR failed completely and returned nothing. A balance of "₦999,999,999,999" is almost certainly an error.



if 1 <= value <= 1_000_000_000:

    return True

else:

    return False  # Suspicious. Re-read.



This catches the edge cases—numbers that pass format and double-read checks but are clearly wrong in context.



The Celebration



After implementing all three layers, I re-ran the 50-test battery. The error rate dropped from 20% to 6%. For the specific "₦15,000" test case, 20 out of 20 reads were correct with double-read confirmation.



The agent now succeeds 19 times out of 20 on multi-app financial tasks. The one failure is a timeout (banking app loads slowly on an old device), not a misread. And critically, when verification fails, the agent now knows it failed. It reports low confidence and aborts the task instead of silently sending wrong data.



What This Taught Me



The bug wasn't in the OCR engine. It was in my architecture. I had built a system that trusted a single, unreliable data source with financial information. That wasn't a Tesseract problem. That was a design problem.



The fix wasn't a better model. It was a better process. Three cheap, fast checks that together create confidence where no single check could. Defense in depth applied to data integrity.



This changed how I think about every feature I build. I now ask: "If this component silently returns wrong data, will I know? If not, where's the verification?"



The Code



The full verification system is open source on GitHub:



👉 github.com/Dexter2344/phone-agent



vision.py contains the verify_financial_data() function. agent.py calls it before storing any financial value in task memory.



The Build Log



I've been documenting this project publicly for 17 days on Dev.to. Every breakthrough, every dead end, every "why is this broken" debugging session at 1 AM.



This bug was Day 17. The one that almost shipped. The one that whispered.



It's not the loudest bug I've ever caught. But it's the one that taught me the most.



This is my entry for the Dev.to Smash Stories contest. The Gemma 4 model powers the agent's command parsing and reasoning. The verification layer is my own work.