A few years ago, I watched a $3M AI initiative get killed in a boardroom. Not because the technology failed. Because nobody could answer a simple question: "How does this make us money?"
The CFO asked it. The room went quiet. The project lead started talking about "strategic optionality" and "future-proofing the value chain." The CFO checked her phone. Two weeks later, the project was dead.
That moment crystallized something I've seen dozens of times: the gap between AI excitement and AI value isn't technical. It's strategic.
The organizations that succeed with AI aren't chasing technology trends. They're systematically addressing bottlenecks in their value chain. They ask 'what problem costs us the most?' before they ask 'what can AI do?'
The Problem I Keep Seeing
Most AI initiatives fail to deliver meaningful ROI. Gartner's research suggests 85% of AI projects fail to reach production. That number feels right. Maybe conservative if you include projects that technically "succeed" but don't deliver the promised value.
But here's the thing: the problem isn't the technology. Machine learning works. Neural networks are powerful. The problem is selection.
Organizations gravitate toward impressive-sounding projects ("Let's build a recommendation engine!" "We need a customer-facing chatbot!") without rigorously evaluating whether those projects address actual business constraints.
I start with financial impact and work backward to technology. What costs you the most money right now? Where are the biggest gaps between what you have and what you need? Which problems, if solved, would meaningfully change your competitive position?
Then, and only then, do we ask whether AI is the right tool.
The Framework That Evolved
The framework I use now isn't the one I started with. The original version had six pillars and took three hours to complete. Nobody finished it.
Version two collapsed it to four pillars with weighted scores. Better, but the weights felt arbitrary. Why 35% for strategic alignment? Because it felt important? That's not good enough.
Here's what actually works: a framework that forces honest answers to uncomfortable questions, not a scoring system that lets people game their way to approval.
The Three Questions That Matter
Question 1: Can you quantify the impact within ±25%?
"We think this will improve customer experience" isn't a business case. "We expect 8-12% reduction in churn based on industry benchmarks and our specific customer dynamics" is a starting point for a real conversation.
If the team can't articulate a reasonable financial impact with some confidence bounds, that's a sign the thinking isn't rigorous enough.
Question 2: Do you have the data, or do you think you have the data?
I've watched projects die because the "readily available data" turned out to require 8 months of data engineering work. The gap between what people believe exists in their systems and what actually exists is enormous.
Question 3: What happens if this takes 14 months instead of 6?
Every timeline I've seen is optimistic. The efficiency gains are often real, but they take 14 months to materialize, not 6. The CFO calculates $1.8M in savings; the plant manager says $2.4M. They're both right, using different baselines.
If your business case only works with the optimistic timeline, it's not a real business case.
The Midwest Manufacturer and Carlos
A heavy equipment manufacturer in the Midwest had been talking about predictive maintenance for three years. Every annual planning cycle, it showed up. Every year, it got pushed.
When I walked their main production floor, the maintenance supervisor pointed to a CNC machine from 1987. "This one talks to us," he said. "The new ones?" He shrugged. "They just break."
That conversation revealed the real blocker. It wasn't data. It wasn't budget. It was that the people who knew the machines didn't trust the sensors, and the people who trusted the sensors had never touched the machines.
We started there. Not with algorithms. With a maintenance tech named Carlos who'd been with the company for 23 years and a young data engineer who'd never seen a factory floor. Their first project together: instrumenting Carlos's 1987 CNC machine and letting him validate the predictions against his intuition.
Six months later, predictive maintenance was working. Not because the AI got smarter. Because Carlos trusted it.
The results? Somewhere between 22% and 35% reduction in unplanned downtime, depending on how you count. The efficiency gains took 14 months to materialize, not 6. The CFO calculated $1.8M in savings; the plant manager said $2.4M. They're both right, using different baselines.
What I Got Wrong
I helped kill a $3M initiative last year. Not reluctantly. Enthusiastically.
A professional services firm wanted to build an AI-powered knowledge management system. The vision was compelling: surface the right expertise to the right people at the right time. They'd done the business case. The numbers worked. The technology was proven.
I recommended they kill it.
Here's why: When I interviewed partners about how they actually found expertise, none of them mentioned the existing knowledge management system. They called people they trusted. They asked their assistants. They searched their email.
The problem wasn't the technology. The problem was that the firm had a trust culture, not a knowledge culture. Building a better search engine wouldn't change that. It would just create a more expensive tool that nobody used.
We redirected the budget toward something smaller: an AI assistant that helped partners prepare for client meetings by synthesizing relevant firm experience. It worked because it fit into an existing behavior (meeting prep) rather than trying to create a new one (using the knowledge system).
The lesson: Sometimes the best AI strategy is a smaller AI strategy.
The Hidden Costs Nobody Budgets For
Most ROI analyses miss these costs beneath the surface that turn profitable projects into money pits:
Data Preparation (60-80% of Effort)
The model is maybe 20% of the work. The other 80%:
- Data discovery: Finding out what data you actually have vs. what people think you have
- Data cleaning: Handling missing values, correcting errors, resolving inconsistencies
- Data integration: Combining data from multiple systems with different schemas, definitions, and update frequencies
- Pipeline development: Building reliable processes to move data from source systems to the ML environment
A rule of thumb: whatever your vendor quotes for the "data preparation phase," multiply by 3-4 for realistic budgeting.
The Operating Cost Surprise
A financial services firm budgeted $2.1M for a fraud detection system. Accurate estimate for build. They didn't budget for the $800K annual operating cost. Model monitoring, retraining, incident response, compliance documentation. That surprise soured the relationship with their AI initiative.
AI isn't "build once, run forever." Models degrade as the world changes.
High-Impact Use Cases That Actually Delivered
Based on projects I've implemented, here are the patterns that consistently work:
Lead Scoring Done Right
A B2B technology client implemented AI-driven lead scoring. The results:
- 23% improvement in close rate
- 31% reduction in sales cycle length
- 40% reduction in time spent on leads that never convert
The model identified patterns across 47 variables that human reps couldn't process. Leads who engaged with three specific pieces of technical content within 48 hours of a webinar had 8x higher conversion rates. No human would have found that pattern.
But here's the thing: We spent 6 weeks on CRM data quality before building any model. We designed the scoring to augment rep judgment, not override it. Reps could see why leads were scored high or low. We started with Marketing Qualified Leads only, proved value, then expanded.
Process Automation That Respects Humans
Beyond simple RPA (which follows rigid rules), intelligent automation handles exceptions and edge cases. A financial services organization automated 73% of their account opening process:
- Document verification using computer vision
- Identity matching across databases
- Compliance screening with nuanced risk assessment
- Exception routing with context for human reviewers
The 27% that still requires humans? Those are genuinely complex cases that benefit from human judgment. The AI handles the straightforward cases quickly, freeing humans for work that matters.
When NOT to Use AI
This might be the most valuable section. AI is powerful, but it's not the right tool for every problem:
Low Volume, High Complexity Decisions
If you make 50 decisions a year that each require deep context and judgment, that's not an AI problem. It's a process and expertise problem.
Insufficient Data
If you have 200 examples of the thing you're trying to predict, you probably don't have enough signal. Do you have thousands of examples? Are they representative of current conditions? Are they labeled accurately?
When Rules Work
If you can write down the decision rules and they work reliably, you probably don't need ML. I've seen organizations build fancy ML models for problems that could have been solved with a well-designed decision tree.
The question to ask: "What does AI offer here that simpler approaches don't?" If the answer isn't clear, start with the simpler approach.
Patterns From Organizations That Win
They start small and scale proven wins. The organizations with the most AI in production didn't start with enterprise-wide transformation. They started with focused pilots, proved value, and expanded systematically.
They invest in data infrastructure before AI projects. Data warehouses, data governance, data quality programs. This foundational work isn't glamorous, but it makes AI projects dramatically more successful.
They accept that most experiments will fail. A 30% success rate on AI pilots is actually quite good. The successful organizations run lots of experiments, kill failures quickly, and double down on winners.
Getting Started
If you're evaluating AI opportunities, here's the approach I recommend:
- Map your value chain and identify where costs accumulate and value leaks
- Quantify the cost of your top 3-5 problems. What would solving each be worth?
- Assess data readiness for the most valuable problems. Do you have the data needed?
- Run a focused pilot with clear success metrics and a 90-day timeline
- Build on proven wins with expanded scope, or kill failures quickly and try something else
The organizations that win with AI aren't necessarily the most technically sophisticated. They're the ones with the discipline to focus on business value rather than technological novelty.
That discipline, the willingness to say "no" to impressive demos that don't solve real problems, is what separates AI success stories from expensive disappointments.