Many AI pilots work and then go nowhere. Nobody owns the result, the real running cost was never counted, the process was not changed, or nothing was standardized, so the next use starts from scratch. Scaling is less about a better tool than about an operating model.
This guide counts the whole cost of ownership with a worked example in which people and monitoring are almost half the running cost; describes an operating model with a sponsor, a governance group, use owners, IT and security, data stewards, an enablement team, and shared standards; gives a five-step maturity ladder and six gates for scaling a use; and explains portfolio review, including retiring uses.
Example data are illustrative.
Before You Start
Why Pilots Stall
Many organizations run pilots that work and then go nowhere, a state sometimes called pilot purgatory. Common reasons are that nobody owns the result, the real cost of running the use was never counted, the process around it was never changed, or the next use must start from scratch because nothing was standardized. Scaling is less about a better tool than about an operating model: roles, standards, and a portfolio view.
Count the Whole Cost
Licences are the visible part. Review, monitoring, and support often cost more.
Standardize What Repeats
Registers, templates, test sets, and review rules make the second use cheaper than the first.
Manage a Portfolio
Decide what to continue, scale, change, or retire, using evidence.
Grow Step by Step
Earn each rung of autonomy and each new site with results.
The Whole Cost of Ownership
Riverside Plant, an illustrative 140-employee metal fabrication and assembly plant, priced its complaint classifier honestly. The licence is $18,000 a year, usage $6,000, review by people $17,500 (a quarter of one person), monitoring and re-validation $8,000, and support and change $5,000, for $54,500 a year. Benefits are $56,000 of analyst time freed and $24,000 of escapes avoided, $80,000 in all, so the net is $25,500 a year. With $30,000 of one-time setup, payback is about 14 months. People and monitoring are 47% of the running cost. A business case that left them out would have shown a net of $51,000 a year instead of $25,500. Figures are illustrative; use the AI Business Case and ROI Calculator for your own.
Watch usage-based pricing. Costs tied to volume or to the number of calls can climb with success, so set a budget and an alert, as you do for the limits on an agent.
An Operating Model That Scales
| Element | What it does |
|---|---|
| Executive sponsor | Sets direction, removes obstacles, and owns the portfolio decisions |
| Governance group | Meets regularly; owns the policy, the use-case register, and the risk tiers; includes operations, quality, IT, security, legal, HR, and safety |
| Use owners | One accountable person for each AI use, from the process, not from IT |
| IT and security | Approved tools, access, logging, integration, and incident response |
| Data stewards | Data definitions, quality, and golden test sets |
| A small enablement team | Templates, training, a prompt library, and help for new uses (sometimes called a center of excellence) |
| Shared standards | The same policy, register, validation method, monitoring, and incident process for every use |
A Maturity Ladder
Do not skip rungs. A register and a policy come before a portfolio review, and monitoring comes before more autonomy. Assess where you stand with the governance guide, and move one rung at a time.
Gates for Scaling a Use
- Results: the pilot met its written criteria, with intervals, against a baseline.
- Cost: the whole cost of ownership is counted and the business case still holds.
- Controls: monitoring, incident response, and a fallback are in place.
- People: the roles are staffed and trained, and the people affected were involved.
- Data and security: data readiness and security checks are passed for the wider scope.
- Repeatability: templates and test sets exist so the next site starts ahead.
Review the Portfolio
Once or twice a year, list every use with its results, cost, risk tier, and incidents, and decide for each: continue, scale, change, or retire. Retire uses that no longer pay for themselves or that carry more risk than value. A portfolio that only grows is a sign that nobody is checking. Use the AI Use-Case Prioritization Matrix to rank new ideas against the live ones.
Common Mistakes
Scaling Before Proving
More sites, same unproven results.
Counting Only the Licence
Hidden review and monitoring cost.
Every Team Its Own Way
No common policy, register, or test method.
No Owner After the Pilot
The champion moves on and the use drifts.
Never Retiring a Use
The portfolio grows and the risk with it.
Autonomy Without Evidence
Moving up a level because it is convenient.
Self-Assessment Questions
- Do we know the full annual cost of ownership of each live use?
- Is there a named owner for each use, from the process?
- Do all uses follow the same policy, register, validation, and monitoring?
- Do we review the portfolio and retire what does not pay?
- Do we know which rung of the ladder we are on?
Scaling AI from Pilot to Portfolio: Frequently Asked Questions
Why do AI pilots fail to scale?
Usually for organizational reasons: no owner for the result, hidden running costs, no change to the surrounding process, and no standard assets such as registers, test sets, and templates. The model rarely turns out to be the problem.
What does an AI use really cost to run?
More than the licence. Include usage charges, the time people spend reviewing output, monitoring and re-validation, support and change work, and the one-time setup. Review and monitoring are often the largest items.
What is an AI center of excellence?
A small team that provides templates, training, a prompt library, test sets, and help to new uses, so each team does not start alone. It supports use owners; it does not replace them.
When should we retire an AI use?
When results fall below the manual process or the tolerance and cannot be recovered, when it no longer pays for itself, when the risk exceeds the value, or when a better approach exists. Retiring on evidence is a sign of a healthy process.
How do we avoid every team doing AI differently?
Use one policy, one register, one validation method, one monitoring approach, and one incident process, owned by a governance group, with room for risk-based variation.
Sources and Further Reading
- National Institute of Standards and Technology, AI RMF 1.0 (NIST AI 100-1, 2023), on governance and the roles involved across the lifecycle.
- ISO/IEC 42001:2023, Information technology: Artificial intelligence: Management system (check the current edition), on a management system approach.
- ISO 9001:2015, clauses 5 and 9.3, on leadership and management review (check the current edition).
- Davenport, T. H. and Ronanki, R., “Artificial intelligence for the real world,” Harvard Business Review, January-February 2018, on projects that start small and tie to business value.