Why AI Models Fail in Enterprise: The 89% Problem

The Vendor Risk Mistake: Why We Built the Wrong Tool

David Ohnstad's team failed Gate 2 and Gate 5 before a single line of code was written. The vendor risk workflow required deterministic outputs with full auditability — exactly the scenario where rule-based systems excel and AI introduces unnecessary complexity. But the product roadmap had "AI-powered risk assessment" as a committed feature for two quarters, driven by competitive pressure and board-level interest in the company's AI strategy. The team built the feature because it was on the roadmap, not because the workflow justified it.

The technical implementation worked. The model ingested vendor questionnaires, security documentation, financial data, and third-party threat intelligence feeds. It output a 0-1 risk score with reasonable precision. The problem surfaced during user acceptance testing: procurement teams couldn't explain why a vendor scored 0.72 instead of 0.68, and they couldn't override the score without escalating to security leadership. The old checklist let them approve low-risk vendors in 15 minutes with a clear paper trail. The AI system required 30 minutes of data entry, produced a score they didn't trust, and forced escalations for edge cases the rules used to handle automatically.

Six months after launch, procurement had informally reinstated the manual checklist for 89% of assessments. They used the AI system only for vendors that triggered automatic escalation flags (high transaction volume, access to sensitive data, geographically distributed infrastructure). For standard SaaS vendors, marketing agencies, and low-risk contractors, the checklist was faster, more transparent, and required less training. The AI feature became a compliance theater checkbox: "Yes, we have an AI-powered vendor risk platform" for RFP responses, but not the actual operational system.

What would David Ohnstad do differently? Ship the rule-based system first. Instrument it thoroughly: track which rules trigger most often, where users request overrides, how often edge cases require manual review, and which vendor categories consume the most assessment time. After 12-18 months of production usage, analyze whether AI could improve the specific bottlenecks you've measured — not the hypothetical ones you assumed existed. If 80% of assessments resolve cleanly with six deterministic rules, and 15% require human judgment on qualitative factors, and 5% involve complex risk modeling across dozens of variables... you've identified where AI might add value (that 5%), and you've avoided building it for the 95% where it creates friction. This approach also builds the labeled training dataset you need: every manually reviewed vendor becomes a training example, and you're collecting data on the actual edge cases that matter, not synthetic scenarios.

Stop Treating AI as a Feature and Start Treating It as a Cost Center

Most product roadmaps list AI features the same way they list UI improvements or API expansions: as discrete capabilities that deliver user value. That framing is wrong for enterprise software. AI isn't a feature — it's a permanent operational expense that compounds over time. Every AI model you ship requires ongoing monitoring, retraining, drift detection, explainability tooling, and incident response when predictions go wrong. According to Forrester's 2024 AI Operations Survey, enterprises spend an average of $180,000 annually per production ML model on maintenance, monitoring, and retraining infrastructure — and that figure excludes the initial development cost.

Rule-based systems have upfront complexity (defining the rules, handling edge cases, building override workflows) but near-zero marginal maintenance cost once deployed. AI systems have back-loaded complexity: they're exciting to build, they demo well, and they fail slowly over months as data distributions shift and model performance degrades. If you can't commit to staffing an ML engineer and a data analyst to maintain the model for the next three years, you're not ready to ship an AI feature — you're accruing technical debt you can't service. provides a deeper strategic framework.

When AI Actually Belongs in Vendor Risk (and It's Not What You Think)

AI makes sense in vendor risk workflows in exactly one scenario: when you're processing continuous telemetry data from vendors post-onboarding, not point-in-time assessments during procurement. Anomaly detection on vendor API usage patterns, network traffic analysis for supply chain security, or behavioral scoring based on vendor support responsiveness and incident disclosure timelines — these are workflows where you're generating thousands of data points per vendor per month, human review isn't scalable, and you're looking for outliers rather than making binary approve/reject decisions.

The product teams shipping . For more on , visit his other sites.