Let me say the thing nobody says in the portfolio review: your AI pilot didn't fail. Failure implies it had a chance. It was designed to produce a demo, not a decision — and on that measure, it succeeded beautifully.
Only 19% of CIOs say most of their AI initiatives met ROI goals. The industry keeps treating the other 81% as a technology problem. It isn't. The models worked. The demos dazzled. What failed was everything around the model — the part nobody designed.
Here's the uncomfortable truth about pilots: a real experiment is designed to be killed. Before it launches, someone writes down what "working" looks like in numbers, what "not working" looks like in numbers, and the date on which the verdict gets delivered either way. That's what an experiment is. What most organizations run instead is a performance — a fixed budget, a friendly use case, a demo date, and an unspoken agreement that success means the demo didn't crash.
And the tell is always the same: ask what the pilot is allowed to conclude. A real pilot can conclude "don't build this." A theater pilot can only conclude "we need phase two." If killing it was never on the table, you weren't running an experiment — you were running a procurement justification with better lighting.
Run it through SPICE:
Strategy. A pilot without kill criteria isn't a strategy; it's a wish with a budget. Before a single line of code runs, write the obituary in advance: under what conditions do we scale this, and under what conditions do we kill it? Put both in writing, signed by the sponsor. If the sponsor won't sign the kill conditions, you don't have a pilot. You have a hobby with headcount.
Politics. And here is why nobody writes the kill criteria: killing a project is a political act. Somewhere in your organization right now is a zombie pilot — everyone on the team knows it's dead, the metrics quietly say it's dead, but the executive sponsor still loves it, because it was their idea, their budget ask, their story for the board. The portfolio review becomes a funeral nobody wants to attend. So the project shambles on, consuming the two scarcest resources you have: engineering time and executive attention. Leaders are now running portfolio reviews specifically to kill underperformers, and that's progress — but a review doesn't kill projects. Incentives do. If the sponsor's standing depends on the pilot surviving, the pilot survives, and the review becomes theater about theater.
Innovation. The irony is that the willingness to kill is what makes innovation work. A portfolio where nothing ever dies isn't a portfolio — it's a museum. The organizations getting real returns from AI aren't the ones with the best models. They're the ones with the fastest funerals: small bets, honest verdicts, resources recycled into the next bet within weeks, not fiscal years.
Culture. Ask what happens to the person who kills their own pilot. In most companies, they get quietly punished — the failed initiative on the performance review, the whisper that they "couldn't deliver." Until killing a dead project is celebrated the way shipping a live one is, your people will keep every zombie alive out of pure self-preservation. And notice who gets blamed when the zombie finally dies of old age: the team that "failed to deliver" — never the sponsor who refused to read the metrics. The people closest to the truth pay the highest price for it, which is exactly why the next team will hide the truth earlier. Culture isn't what your AI strategy deck says. It's what happens to the person who brings bad news early.
Execution. So here's the machinery: every pilot gets a written hypothesis, a success metric, a kill metric, a verdict date, and a named owner for the verdict — and the owner cannot be the sponsor. Separate the advocate from the judge. When the date arrives, the verdict is delivered in the open, the resources are reallocated within thirty days, and the team is publicly thanked for the learning. Do that five times and watch what happens to your sixth pilot: suddenly everyone designs for the verdict, because they know the verdict is real.
Today, most AI pilots are designed to never end — no kill criteria, no verdict date, no judge. Tomorrow belongs to the organizations that treat every pilot as guilty until proven innocent, and say so out loud before the money is spent.
Stop asking why your pilots fail. Start asking who benefits from them never ending.
Leave a Reply