The pricing formulas in Motor, the estimating engine I built for a water feature shop, did not come from the manual. I pulled 32 of them out of the JavaScript behind Aquascape's contractor calculator, the tool contractors actually use to bid jobs. The manual was sitting right there, official and free. Ignoring it was the best design decision in the whole system.
A vendor never ships a sloppy calculator
Why trust the calculator over the manual? Because of what happens when each one is wrong. If the manual sizes a pump wrong, a reader shrugs and moves on. If the calculator sizes a pump wrong, a contractor bids a job at that number, wins it, and loses money on the install. Then the phone rings. So calculators get fixed and manuals drift. Give it ten years and the two quietly disagree, and everyone in the trade knows which one to trust without anyone saying so.
A vendor will ship a sloppy PDF. They will never ship a sloppy calculator.
Documentation is what a domain says about itself. The artifacts money flows through are what it actually believes. Once you see that split, you cannot stop seeing it.
The other half was in old invoices
Formulas only get you to cost. What a shop charges on top of cost is a belief about its market, and no vendor document holds that number. So I pulled 132 historical quotes out of the shop's CRM. Real quotes, sent to real customers, most of them paid. I calibrated Motor's markup against those, then checked its output against what the shop had actually charged.
The result: Calibrated against 132 real quotes, Motor's estimates landed within 5 percent of what the shop actually charged, with no pricing rule taken from documentation.
I could have just asked the owner what his markup was. But what an owner says and what his invoices show are rarely the same number, and the invoices are the ones customers paid. When the two disagree, believe the invoices.
The same bug in a different industry
I build and run systems in several industries, and the surprising part is how often the same small idea matters in all of them. This year it showed up in medical billing. I run a denial engine that reads insurance denials and recommends the next move: appeal, fix the coding and resubmit, bill the patient, or write it off. The first version reasoned from guidance prose, appeal-strategy text that reads like documentation. On a golden set of 30 adjudicated cases, it picked the right action 36.7 percent of the time.
What broke: The engine kept recommending appeals for coding errors that should just be fixed and resubmitted, because the guidance said appeal and the model believed it. It scored 36.7 percent on action selection. I had banned documentation from my inputs, then smuggled it back in as a prompt.
The fix was the same move as the calculator. We keep a reference layer that records what experienced billers actually do with each denial code, distilled from real worked claims. I anchored the engine to that instead of the prose. Action accuracy went from 36.7 percent to 76.7 percent in one evening, and classification went from 80 percent to 93.3 percent. The model did not get smarter. The spec got real.
- 32 formulas pulled from the contractor calculator's JavaScript
- 132 real quotes used to calibrate markup
- 5% gap between Motor's estimates and actual shop pricing
- 76.7% denial-action accuracy after anchoring to worked claims, up from 36.7%
Two records
Every domain keeps two records of itself. One is written for readers: manuals, best-practice guides, onboarding docs. The other is written for money: calculators, invoices, worked claims. The first record is what the domain wants to be true. The second is what it paid to learn. When you encode a domain into software, you are choosing which record to believe, whether you know it or not.
Key insight: Encode the rules a domain risked money on (the shipped calculator, the paid invoice, the worked claim) and treat everything it merely wrote down as a rumor.
- Pull your rules from the artifacts people bid with. The calculator beats the manual it shipped next to.
- Calibrate against money that actually moved, and measure the gap in percent, not in vibes.
- Treat documentation as a hypothesis about the domain, never as the spec.
- When your model gets a domain wrong, check what you fed it before you blame the model.
None of this is really about ponds or insurance claims. It is about where truth lives in a domain. A domain will tell you anything in its documentation. What it bids with is what it believes. Build from that. And if you are pulling formulas out of someone's calculator at midnight, in an industry nothing like mine, we should compare notes.
Originally published at nabbilkhan.com.
SOCIAL SHARE CARD GENERATOR