Enterprise Impact of the AI Fever: What Almost Four Years of GenAI Have Actually Bought

The story is the gap

Almost four years after ChatGPT shipped on 30 November 2022, the most important enterprise number is not a benchmark score or a model price. It is the gap between what boards have authorised and what front-line workflows have actually absorbed.

McKinsey's State of AI 2025 (published November 2025, n=1,993, fielded June-July 2025) is the cleanest single read on that gap. 88% of organizations now report regular use of AI in at least one business function, up from 78% a year earlier. But only about one-third have begun to scale AI across the enterprise. Only 39% of respondents attribute any EBIT impact to AI use, and most of those say it is less than 5%. Only ~6% qualify as McKinsey "AI high performers" attributing more than 5% of EBIT and significant value to AI. The single strongest leadership predictor of high-performer status is fundamental workflow redesign: high performers are nearly 3x more likely than peers to have redesigned a workflow around AI.

The MIT NANDA report, The GenAI Divide: State of AI in Business 2025, sharpened that picture in July 2025. Despite an estimated $30-40 billion of enterprise spend on generative AI, MIT estimates that roughly 95% of corporate generative-AI pilots are still failing to deliver measurable P&L returns. The gap is concentrated in custom internal builds rather than purchased copilots. IBM's 2026 CEO Study, looking at the leadership layer, found 76% of organizations have appointed a Chief AI Officer and 64% of CEOs are comfortable making major strategic decisions on AI-generated input, yet only 25% of the workforce uses AI regularly in daily work and 83% of CEOs say success now depends more on people's adoption than on the technology itself.

That is the story. AI is now nearly universal in enterprise use, but only shallowly integrated. The firms compounding value are not the ones with the loudest narrative. They are the ones converting model capability into measured workflow change, controlling cost through architecture, and governing AI as a production system rather than a demo.

My operator read: when an AI programme cannot name the workflow owner, baseline, failure mode, and cost guardrail, it is still a demo even if the interface is in production. The organizations that create value make those four names boringly explicit before they ask for another pilot.

This piece is my synthesis of the evidence available up to 12 May 2026: McKinsey's State of AI 2025, MIT NANDA's GenAI Divide, IBM's 2026 CEO Study, the Stanford HAI AI Index 2025, Gartner forecasts, vendor pricing pages, public-company filings, official case studies, and standards guidance. I write it the way I read the market: numbers first, story second, opinion last.

What enterprises are actually building

The deployment mix is far narrower than the rhetoric of "AI transformation" suggests. The dominant patterns are copilots for office productivity, enterprise search on internal content, customer-service assistants, code assistants, document and meeting summarisation, and bounded agents that take actions only inside a workflow shell. Text generation is still the dominant modality. Code and images sit materially behind it.

A practical taxonomy:

The architecture pattern that now dominates production "copilots" is not "one giant model". It is a stack: identity and access control, orchestration, retrieval-augmented generation, tool or API calling, observability, evaluation and human escalation. Microsoft, AWS, Google Cloud and OpenAI all publish near-identical reference architectures. OpenAI's own enterprise guide explicitly recommends starting with a single agent and adding tools incrementally rather than jumping directly to multi-agent systems.

Three technical decisions separate pilots from production. First, RAG for volatile or permission-sensitive knowledge, because it grounds answers on current company data without retraining the model. Second, tool use only where there is a formal workflow boundary, such as CRM updates, ticketing or meeting-note insertion. Third, fine-tuning or distillation reserved for high-volume, stable tasks where prompt-only control is no longer good enough.

The Morgan Stanley pattern is the clearest public example. The firm's early deployment focused on internal knowledge assistance and later on meeting debriefing, with an evaluation framework used to validate output quality before broad production use. By April 2025, public reporting from Morgan Stanley and OpenAI showed 98% daily adviser usage and a jump in document access from 20% to 80%, with natural-language querying over more than 100,000 proprietary research documents. That is not AI replacing the adviser. That is AI acting as a search, summarisation and drafting layer on top of existing human judgement.

Where to apply this on your own portfolio

If you want to pressure-test a specific initiative through the same six layers I use when I write about this (Baseline, Architecture, Surface, Intelligence, Containment, Scale), the diagnostic below is the interactive version of the framework I keep coming back to in this piece.

The economics of tokens, APIs and cloud spend

The pricing market has matured into several distinct models: direct token metering, seat licences, priority or reserved throughput, and platform-layer charges for routing, search, tools or governance. The core point is that a quoted "model price" is rarely the full delivered price.

Vendor pricing comparison

Two strategic observations follow. Vendor list prices are converging downward for limited text-only work, especially for smaller models, but remain structurally asymmetric on output tokens. And the most important cost-control levers are now architectural rather than negotiational: caching, batching, routing simpler prompts to smaller models, and limiting unnecessary output length.

Three workload shapes, priced at list

The next table is intentionally operational. It shows what three common workload shapes look like priced only at current public text-token rates, excluding storage, search, seat licences, security tooling, observability, engineering labour and taxes.

These figures are illustrative, not contractual. They are useful for one insight. For many internal text workloads, model metering alone is not the dominant cost line. A 300-seat Microsoft 365 Copilot Business deployment at the annual-plan list price would cost roughly $5,400 per month before implementation work, already higher than the model meter for many modest internal API workloads. Once an organisation runs very high-volume support or consumer-facing workloads, or pushes into video, image, long-context or tool-heavy pipelines, inference becomes materially visible.

Throughput and quotas as real consumption signals

Because public token-volume disclosures are scarce, the best observable indicators of real enterprise usage are throughput quotas, conversations, documents and user adoption. OpenAI lists GPT-5.5 rate limits from 500,000 TPM at Tier 1 to 40,000,000 TPM at Tier 5. Anthropic measures rate limits in RPM, ITPM and OTPM, and notes that with a 2,000,000 ITPM limit and an 80% cache-hit rate, effective input throughput can reach 10,000,000 tokens per minute because cache reads do not count toward ITPM for most current models.

On platform layers, the numbers are larger still. AWS documents 6,000,000 cross-region TPM for Claude Sonnet 4.6 and 15,000,000 for Claude Opus 4.7 on Bedrock. Google Cloud documents 30,000 RPM per model per region as a system limit in Vertex AI, with examples such as 3.4M TPM per project for Gemini 2.0 Flash text workloads and 67,250 tokens per second of continuous throughput from 25 GSUs of provisioned Gemini 2.5 Flash. Azure frames quotas in TPM and RPM per region, subscription and model or deployment type, and positions PTUs as the predictable-capacity option for steady workloads.

Cost increases that are partly attributable to AI

The bluntest macro indicator is the IBM finding that the average cost of compute is expected to rise 89% between 2023 and 2025, with 70% of executives calling generative AI a critical driver. Strong directional signal, still survey evidence rather than audited accounts.

The public-company evidence is more precise but more nuanced: AI costs are usually visible, but mixed with hosting, support and platform growth rather than cleanly isolated.

The conclusion: AI-attributable cost is often only partly isolatable in public reporting. In production, AI cost arrives as a bundle: model inference, GPU-heavy cloud workloads, enterprise search, data engineering, security tooling, support and adoption programmes. This is precisely why AI economics can look cheap in a lab and expensive in a finance review.

Business outcomes and ROI

The best evidence does not show a uniform AI productivity dividend. It shows bounded gains in bounded contexts, with the biggest pay-offs where the task is frequent, the workflow is measurable, and the error surface is controlled.

At the market level, the most-quoted ROI multiples are still primarily survey- or sponsor-based and should be read as ambition-setting rather than audited returns. The credible counterweight is MIT NANDA's GenAI Divide finding (July 2025) that roughly 95% of generative-AI pilots are still failing to produce measurable P&L impact. Both numbers can be true at the same time: a small minority of firms are extracting outsized returns; the modal firm is not.

The sober ROI rule: AI creates measurable value when time saved is actually redeployed into throughput, sales, service or accuracy, not when the organisation simply observes that people finished a task faster. McKinsey 2025's finding that workflow redesign is the single strongest leadership predictor of high-performer status is the best one-line summary of this point.

Failure modes, hidden costs and organisational effects

The most expensive enterprise AI mistakes are usually not model mistakes. They are system-design and operating-model mistakes.

Hidden technical costs

The first is token inflation outside the base prompt. Tool schemas, system prompts, retrieved chunks, conversation history and model-specific tokenisers all change real spend. Anthropic notes that Opus 4.7 may use up to 35% more tokens for the same fixed text because of a new tokenizer; tool-use documentation shows extra system-prompt overheads of roughly 313-346 tokens per call once tools are enabled. A headline "$5 per MTok" can materially understate actual cost in a tool-heavy, poorly pruned implementation.

The second is latency as a function of verbosity. Google Cloud states explicitly that latency is directly proportional to generated token length and recommends constraining max_output_tokens to avoid needless delay and spend. Long outputs increase direct inference cost, queueing pressure, user wait time and reviewer burden.

The third is retrieval and data engineering debt. Production RAG means document ingestion, chunking, indexing, permissions replication, hybrid retrieval tuning, citation validation and connector maintenance. Azure, AWS and Google all treat search, document extraction and knowledge-layer design as first-class architecture components.

Quality, security and legal risk

Quality risk still centres on inaccuracy and hallucination, followed by privacy, cybersecurity and IP. McKinsey's State of AI 2025 reports that organisations are increasingly investing in mitigations for inaccuracy, cybersecurity and intellectual-property risk, but most still rely on a small set of controls. Independent reporting on production deployments shows that hallucinations, prompt injection and sensitive-data leakage remain the dominant failure modes when LLMs are exposed to user input or tool use without strong evaluation harnesses.

Standards bodies now treat this as a core management issue. NIST's AI RMF Generative AI Profile is the companion to AI RMF 1.0 for identifying and managing trustworthiness risks in generative-AI systems; SP 800-218A extends secure software development practices to generative AI and dual-use foundation models. The EU AI Act's Article 50 transparency obligations begin to apply on 2 August 2026, requiring providers and deployers to label AI-generated content and disclose interactions with AI systems. Together these frame oversight as a production discipline, not a prompt filter.

Organisational effects

The organisational impact is already material. IBM's 2026 CEO Study reports that 76% of organisations have appointed a Chief AI Officer, with stronger CHRO involvement and broad expectations for reskilling. McKinsey 2025 finds that the rise in AI roles is concentrated in technical and product profiles, while AI compliance and ethics specialists remain rare relative to the scale of risk and compliance work being centralised. That capability gap is one of the better explanations for why so many programmes stall at the governance layer rather than at the model layer.

Vendor lock-in is also deeper than many procurement teams first assumed. Prompt portability is relatively easy. Operational portability is not. The moment an organisation relies on Azure PTUs, Google GSUs, Anthropic-specific cache semantics, Bedrock routing tiers or proprietary SaaS-grounding layers, the switching cost moves from prompt text to runtime economics, observability, approval logic and governance. Anthropic's own documentation notes that its models run across the Claude API, Bedrock, Vertex AI and Microsoft Foundry, with differing regional and pricing treatments.

Shadow IT is no longer just a security issue. It is a procurement and FinOps issue. When secure chat is included in an existing suite, when agent builders are sold as low-code tools, and when model metering looks cheap in isolation, unmanaged sprawl becomes easy. Without central gateways, workspace limits, budgets and audit logs, organisations end up paying for duplicate copilots, duplicate connectors and duplicate review processes.

Governance and the AI FinOps playbook

The evidence points to a fairly clear set of practices.

First, establish a central AI control plane for data governance, model access, observability and policy, while leaving business-unit experimentation partly decentralised. This matches how risk and compliance are typically centralised while adoption and talent are more often hybrid.

Second, treat use-case selection as a portfolio problem. The best first-wave candidates are high-frequency tasks with manageable downside, clear baselines and measurable outputs: support draft responses, repo-grounded coding help, internal knowledge retrieval, meeting notes, document summarisation. Start with a single agent and add tools incrementally.

Third, build AI FinOps into delivery from day one. At minimum:

• output caps and verbosity control;
• prompt caching for repeated instructions and long context;
• batch processing for asynchronous jobs;
• routing simple requests to smaller models;
• separate budgets for model spend, search/indexing and human review;
• chargeback by business unit or workspace.

Fourth, require evidence of value, not just adoption. The right unit of account is the business process: resolution time, merge rate, cycle time, campaign throughput, gross margin, conversion, claims accuracy or working-capital velocity. Vendor-sponsored ROI multiples are useful for ambition; internal ROI should be measured against holdouts, pre/post baselines or randomised roll-outs wherever possible. The GitHub/Accenture study is a good model because it combines telemetry, user surveys and an explicit experimental design.

Fifth, tier human oversight by risk. External-facing or regulated content should have stronger review, provenance and escalation rules than low-risk internal drafting. NIST's AI RMF Generative AI Profile and SP 800-218A, plus the EU AI Act's Article 50 obligations from 2 August 2026, provide the best public templates for making oversight operational rather than aspirational.

Open questions

Three questions remain structurally hard to answer from public data. The first is exact enterprise token spend by customer: public disclosures remain sparse, so any cross-company token benchmark is still mostly inferential. The second is true all-in ROI: many vendor case studies disclose benefit but not the full cost stack, while many filings disclose cost pressure without a clean process-level counterfactual. The third is durability of uplift: short-run productivity gains are increasingly well evidenced, but the medium-run effects on error rates, organisational design, procurement consolidation and labour mix are still evolving. The Klarna May 2025 reversal and Duolingo's gross-margin swings show the economics can move quickly in both directions as deployment quality and unit costs change.

The highest-confidence conclusion, almost four years in, is already clear. The enterprise impact of the AI fever has been real but uneven. The most successful firms are not the ones with the loudest AI narrative. They are the ones converting model capability into measured workflow change, using architecture to control cost, and governing AI as a production system rather than a demo.

References

• McKinsey & Company. The state of AI in 2025: Agents, innovation, and transformation. November 5, 2025. n=1,993.
• MIT NANDA. The GenAI Divide: State of AI in Business 2025. July 2025.
• Stanford HAI. Artificial Intelligence Index Report 2025.
• IBM Institute for Business Value. 2026 CEO Study.
• Gartner. Predicts 30% of Generative AI Projects Will Be Abandoned After Proof of Concept By End of 2025. July 29, 2024.
• Gartner. Forecasts Worldwide GenAI Spending to Reach $644 Billion in 2025. March 31, 2025.
• OpenAI. API pricing, rate-limit and enterprise documentation. platform.openai.com (captured May 2026).
• Anthropic. Claude API pricing, prompt caching and tool-use documentation. docs.anthropic.com (captured May 2026).
• Google Cloud. Vertex AI and Agent Platform pricing (Flex/Batch, Priority, GSUs).
• Microsoft. Azure OpenAI pricing, PTUs, and Microsoft 365 Copilot Business pricing.
• Amazon Web Services. Amazon Bedrock pricing across Standard, Flex, Priority and Reserved tiers.
• Morgan Stanley. Launch of AI @ Morgan Stanley Debrief; OpenAI customer story on Morgan Stanley evals.
• GitHub with Accenture. Enterprise Copilot deployment study.
• Bloomberg. Klarna Turns From AI to Real Person Customer Service. May 8, 2025.
• Reuters. Reporting on Klarna CEO commentary on the limits of AI cost-cutting in customer service, 2025.
• Duolingo. Quarterly gross-margin disclosures, FY2025 to Q1 2026.
• Snowflake. FY2026 filing on cost of product revenue.
• NIST. AI Risk Management Framework Generative AI Profile and SP 800-218A.
• European Commission. AI Act, Article 50 transparency obligations applicable from 2 August 2026.