The pilot went well. The team is pleased, the numbers are good, the committee approves the rollout.
Six months later the rollout is stalling and nobody understands why.
The answer is almost always the same. The pilot did not lie. It simply measured something other than what everyone thought it was measuring.
What the pilot does not reproduce
The people. A pilot runs with volunteers. They are the most curious, often the most capable, and they want it to work. The rollout runs with everybody, including those who asked for nothing and those who have watched three tools come and go in five years. Different population, different result.
The data. A pilot runs on a chosen perimeter, therefore a clean one. The rollout meets the old databases, the duplicates, the fields filled in by hand, the exceptions nobody had documented. What took two minutes now takes twenty.
The backing. During a pilot, the leadership is watching. Blockages surface fast and get cleared fast. By rollout, attention has moved to the next subject, and the same blockage takes three weeks.
What makes a pilot useful
A pilot is there to learn, not to convince. The difference shows in one thing: is it known in advance what would make it stop?
Before launching, put three things in writing.
What this pilot must demonstrate, as one precise measure. What it cannot demonstrate, and will have to be tested another way. The result below which you walk away.
The third is the hardest to write and the most profitable. Without it, no pilot ever really fails. They get extended, widened, renamed. And the organisation accumulates experiments that never end.
The real cost
The money spent comes back. The credibility much less so.
Every project launched with fanfare and then dropped without explanation teaches your teams one thing: wait. After three cycles, the best strategy for an employee becomes to change nothing, since it will pass anyway.
Count the experiments open in your organisation right now. Then count the ones with an end date written down somewhere. The gap between those two numbers will tell you where you stand.
Related topics