I began the earlier version of this essay in a Zurich hotel room, reading confident predictions that AI was a bubble about to burst. Part of me wanted the capability curve to flatten so we could all catch our breath.
The argument was appealing because it joined several concerns that deserve attention: large investment, uncertain returns and demonstrations that work better than real operations. But joining them into a single verdict made the next decision harder. A technology can improve while a supplier overinvests, a valuation falls or a particular deployment loses money.
The question I now find useful is whether our organization can learn from improving capability without committing itself to economics that have not been demonstrated.
Three claims that need separate evidence
The first claim concerns what a model can do. Benchmarks can establish progress on defined tasks under stated conditions. They do not establish what happens in a company's workflows.
The second concerns whether a deployment can perform reliably enough to be useful. It depends on sources, integrations, permissions, exceptions and the consequences of a wrong action. A better model can help, but it does not settle those operating questions.
The third concerns financial return. A useful deployment may still cost too much, arrive too late or save capacity that nobody can turn into revenue or lower expense. Conversely, a modest assistant can create value without being near the frontier.
Each claim needs its own evidence. Treating one benchmark score as proof of all three turns a capability observation into a business promise.
What the task-horizon evidence says
METR's January 2026 update measured the human-equivalent task length at which models succeeded half the time on its evaluation suite. Its fitted growth trend depended on the period examined: roughly 196 days to double across the stitched historical record, and 89 days for the post-2024 subset. The researchers explicitly describe sensitivity to task composition and wide uncertainty for long tasks.
That is evidence of rapid progress in the evaluated work, not a promise that enterprise productivity doubles on the same schedule. Fifty-percent success is also a very different standard from the reliability required to release a quote, update a financial record or control equipment.
The implication I draw is organizational: evaluation methods should be reusable so that a changed model can be tested without rebuilding the whole approval process. This is my operating judgment, not a result measured by METR.
A procurement decision with two clocks
Imagine an equipment supplier considering an assistant for technical quotations. This is a worked scenario, not a report of an engagement. The team wants to reduce time spent finding specifications and preparing a first draft. It is not proposing autonomous release of commercial commitments.
One clock measures model change. New versions may improve document interpretation or drafting. The other measures the supplier's ability to introduce a dependable workflow: source control, data access, engineering review, user training and ownership after launch.
Waiting for the model market to settle does not finish the second clock's work. Buying a long contract immediately does not finish it either. The company can assemble a small set of completed quotations, define the evidence each answer must cite and test whether the proposed assistant improves the actual preparation-and-review process.
The test should include awkward cases: an obsolete specification, conflicting customer requirements, a missing drawing and a price that needs approval. A result that looks fluent but cites the wrong equipment revision is a failed result. Review effort counts as part of delivery, rather than being written off as a temporary inconvenience.
Once those checks exist, another model can run against the same cases. The team has built an asset that remains useful when supplier claims change.
Preserve the ability to change your mind
The investment choice should distinguish work worth doing regardless of the model from commitments that depend on a particular supplier's performance. Cleaning a critical product record or naming the quote-release owner may have value under several outcomes. An exclusive agreement, deeply proprietary integration or large deployment before measured use is a less reversible bet.
This does not mean every decision must be small. Some opportunities justify material commitments. They should have a clear explanation of what has been learned, what remains assumed and what would cause the company to stop or change course.
For the illustrative quotation assistant, I would look for exportable source records, a defined exit process and a way to retain the evaluation cases. I would also ask how performance changes are introduced. A supplier should not quietly change the operating behavior on which a release decision depends.
Optionality has a cost too. Running several platforms indefinitely, duplicating reviews and postponing all standardization can consume the gain. The objective is enough flexibility to respond to evidence, with a date and owner for choosing the operating approach.
What would make me decline the investment
I would decline a production commitment if the team could not show which workflow improves, if a reviewer could not check consequential claims, or if the proposed saving depended entirely on ignoring review and correction. I would also question a system whose value case required a forecast about capabilities that have not yet arrived.
Those are useful grounds for a decision because the company can investigate them. Whether the entire AI market deserves the label “bubble” is much less actionable for a business owner trying to release dependable work.
The answer may still be to wait on a specific deployment. Waiting can be disciplined when the missing evidence is named and someone is gathering it. It becomes expensive when it substitutes for learning about the process the company will eventually need to change.
Capability progress deserves attention, and investments deserve separate scrutiny. For the quotation team, that means keeping the test cases ready while deciding which operating commitment the evidence can support. A changing model market makes that judgment more useful, not less.