Your AI pilot succeeded and it told you nothing
A pilot run on curated data, with the team watching, against no baseline, is a demonstration. It predicts almost nothing about production behaviour — and the gap is where most AI budgets go.
Insights
Written by the consultants and engineers on the engagements — including the approaches that did not work. No ghostwritten thought leadership.
9 articles, page 1 of 1.
A pilot run on curated data, with the team watching, against no baseline, is a demonstration. It predicts almost nothing about production behaviour — and the gap is where most AI budgets go.
Replacement is the default recommendation because it is the easiest to scope and the easiest to sell. It is frequently the wrong one, and the analysis that shows this takes about four weeks.
If the handover plan starts after the build, the system has already been designed to need you. Everything about how it was built assumed a team that was there for the decisions.
A retrieval system that returns the wrong passage produces a confident, well-written, wrong answer. Nothing in the output signals the failure, which is what makes it the dangerous kind.
The streaming architecture is a week. Deciding that customer 4471 in one system and CUST-4471-B in another are the same customer is the other five months.
Full decomposition is a multi-year programme that usually stalls halfway, leaving two architectures to operate. Extracting the components that actually hurt is a quarter and it finishes.
Discovery that bills by the day and ends when the budget does is not discovery. It is the first phase of the work it recommends, which is a conflict of interest wearing a deliverable.
An alerting system is only as good as the trust of the team receiving it. Trust is spent by false positives and it does not refill on its own.
Every overrun programme contains a moment, early, where somebody knew. The cost of raising it then and the cost of raising it at month six differ by an order of magnitude.