Why Isn't AI Delivering the ROI We Were Promised?

Perhaps the problem isn't the technology, but how organisations are approaching AI deployment.

There is tremendous excitement surrounding Artificial Intelligence. Organisations are purchasing AI licences, experimenting with AI agents and exploring opportunities to automate work. The promises are compelling: greater productivity, lower costs, faster processes and potentially fewer employees.

Yet the returns have been less convincing.

A widely discussed 2025 MIT Project NANDA report suggested that approximately 95% of the enterprise Generative AI initiatives it examined were not delivering measurable financial returns. While this should not be interpreted as a failure rate for all AI projects, it raises an important question: If AI is so powerful, why are organisations struggling to demonstrate its business value?

I suspect much of the problem lies not with the technology, but with how organisations approach its deployment.

Buying AI isn't the same as having an AI strategy

Consider two increasingly familiar approaches.

The first is to purchase AI subscriptions, encourage employees to experiment and hope that productivity improves. Some become enthusiastic users, while others struggle to find meaningful applications. There may be genuine benefits, but without clear objectives or baseline measurements, it is difficult to establish how much value the organisation has actually gained.

The second approach is more ambitious. Convinced by the promise of AI agents, organisations attempt to automate entire functions or replace substantial numbers of employees, only to discover that real business processes are considerably more complicated than the demonstrations suggested.

AI may handle routine situations impressively but struggle with exceptions, changing requirements or tasks where mistakes are unacceptable. Additional resources are then needed to supervise, verify, troubleshoot and maintain the system. In some cases, these costs could exceed the savings AI was supposed to deliver.

Both approaches share a fundamental weakness: They start with AI rather than the work that needs to be done.

AI should start with the task, not the job

Jobs are rarely made up of a single task. Consider an HR executive who prepares reports, answers employee enquiries, interprets policies, advises managers and resolves sensitive employee issues.

Some of these tasks could be automated, others might benefit from AI assistance, while some are better handled by experienced employees. The opportunity is therefore not necessarily to replace the HR executive, but to identify specific tasks where AI can improve speed, cost, quality or capacity.

Of course, extensive automation may eventually change staffing requirements or eliminate certain roles. But the starting point should be the task and the value created, not the number of jobs we hope to replace.

At FYT Consulting, we approach AI deployment through three stages: Design, Build, and Decide & Deploy.

Each stage addresses a different question and helps organisations make more informed investment decisions.

Stage 1: Design — Identify the right task and technology

Before developing an AI solution, organisations need to establish exactly what they want to accomplish. What task are we trying to improve? Is the objective faster turnaround, lower costs, better quality or increased capacity? And importantly, is Generative AI even the appropriate technology?

One useful starting point is to distinguish between two broad types of tasks.

Tasks with one correct answer, such as calculating salaries, determining annual leave balances or computing employee turnover rates, require accuracy, consistency and the ability to trace errors. Traditional software, established formulas and rule-based automation are often better suited to these requirements because their outputs are predictable and reproducible.

Generative AI, by contrast, is probabilistic. It can misunderstand instructions, select inappropriate information or produce incorrect answers with considerable confidence. Where mistakes have serious consequences, even occasional errors may be unacceptable.

Tasks with multiple acceptable answers, such as drafting customer service responses, preparing marketing copy or summarising meetings, are often more suitable for Generative AI. Quality depends on relevance, clarity and usefulness rather than matching one exact answer. An imperfect first draft can be reviewed and refined, potentially saving considerable time.

However, this distinction is not absolute. A customer service response involving contractual commitments could have serious consequences if incorrect, while AI can still assist with precise calculations by retrieving results from established systems.

The more important consideration is how much variation and error a task can tolerate, how easily mistakes can be detected, and what happens when something goes wrong.

Once a suitable task has been identified, organisations must establish how it is currently performed. How long does it take? How much does it cost? How often do mistakes occur, and what constitutes an acceptable outcome?

Without this baseline, there is no objective way to determine whether AI has actually improved anything.

Stage 2: Build — Develop, test and improve

This is where organisations often underestimate the work involved. Developing a reliable AI workflow can require considerably more than writing a few clever prompts. It may involve preparing reference information, connecting business systems, defining operating rules, setting boundaries and establishing procedures for handling exceptions.

Most importantly, it requires repeated testing and refinement.

At FYT, we advocate evaluating AI workflows against clearly defined success criteria using representative examples of actual work. Does the AI consistently produce acceptable outcomes? What kinds of mistakes does it make, how frequently do they occur, and what are the consequences?

Consider a customer service executive who takes 10 minutes to prepare a response. If AI reduces drafting time to two minutes but requires another six minutes of checking and correcting, the actual saving is only two minutes. If occasional mistakes lead to complaints or costly follow-up work, even that saving may disappear.

Each iteration should therefore aim to improve performance, whether through faster execution, better quality, fewer errors or lower operating costs. Some approaches may prove too expensive or unreliable and should be reconsidered rather than pursued simply because resources have already been invested.

The objective is not to make AI work at all costs, but to establish whether it can perform the task reliably and economically.

Stage 3: Decide & Deploy — Does the business case make sense?

Once the workflow has been developed and evaluated, management should have sufficient evidence to make an informed decision.

How much faster is it compared with the original process? Has quality improved? What is the cost per successfully completed task? How frequently do errors occur, and what does it cost to detect and correct them?

These measurements provide a more meaningful basis for assessing ROI than simply counting the hours AI appears to have saved.

The hidden costs of AI deployment

The cost of AI goes well beyond subscriptions and initial development. Organisations need to consider several less obvious expenses, particularly when moving from a successful pilot to organisation-wide deployment.

• Making AI reliable. Unlike conventional software that follows established rules, Generative AI can produce inconsistent or incorrect responses. Organisations may need employees to verify outputs, investigate errors and intervene when the AI encounters situations it cannot handle. The cost of these additional checks, and the consequences of undetected mistakes, can significantly erode productivity gains.

• Building the supporting infrastructure. AI rarely operates effectively in isolation. It may need to connect with existing business systems, retrieve reliable information and follow agreed business rules. For analytics applications, this could involve developing and maintaining semantic models that define what the data means and how calculations should be performed. These require business expertise, technical development, testing and ongoing maintenance, even where existing analytical models can be reused.

• Keeping pace with business changes. Organisations restructure, policies evolve and reporting requirements change. AI workflows, reference information, business rules and system integrations may need to be updated accordingly. Every significant change could require further development, testing and employee training to ensure the solution continues to work as intended.

• Managing technology changes and dependencies. AI models are regularly upgraded, retired or replaced, sometimes changing how previously reliable workflows behave. Vendors may alter pricing, product offerings or ownership, while service outages can disrupt AI-dependent operations. Organisations may need to retest applications, rebuild integrations, maintain backup processes or migrate to alternative providers.

• Developing and retaining the necessary talent. Reliable AI deployment requires more than employees who know how to write prompts. Organisations may need people who can design workflows, manage data connections and business rules, evaluate AI performance, troubleshoot failures and maintain the systems. These capabilities must be developed internally, recruited or outsourced, with implications for both cost and organisational dependency.

• Redesigning work and developing employees. As AI takes over specific tasks, workflows, responsibilities and job requirements may need to change. Employees will need training to work effectively with AI, exercise oversight and focus on higher-value activities. These transitions take time and resources, and the anticipated savings may not materialise unless the organisation actually redesigns how work is performed.

• Not all these costs are unique to AI, nor will every deployment incur them to the same extent. But they should be considered when comparing AI against existing tools and alternative approaches.

The true cost of AI is not simply what it takes to get the system working, but what it takes to keep it reliable, relevant and economical as the business and technology evolve.

An AI solution that looks attractive during a three-month pilot may therefore look considerably different when evaluated over three to five years. Management needs to consider the full lifecycle costs, benefits and risks before deciding whether to proceed, expand or abandon the proposed solution.

AI transformation is a journey, not a big bang

Much of the excitement surrounding AI centres on dramatic productivity gains, sometimes accompanied by equally dramatic workforce reductions. But meaningful AI transformation may look considerably less spectacular.

At FYT, our approach is deliberately iterative. Start with a specific task, design an appropriate solution, build and test it, evaluate the business case, and deploy where justified. Then continue measuring and improving.

Each successful iteration may improve performance, reduce errors, lower operating costs or expand the range of tasks that can be automated. The returns from AI may therefore emerge progressively, through many small improvements, rather than one spectacular breakthrough. And as workflows evolve, so will jobs.

Consider our HR executive again. Initially, AI might help draft routine communications. With further improvements, it could handle common enquiries, retrieve information from HR systems or prepare standard reports.

Over time, the employee may spend less time on routine administrative work and more time resolving complex cases, advising managers and addressing situations requiring empathy, judgement or accountability.

The job has not necessarily disappeared. Its composition has changed, and so have the capabilities required to perform it.

Employees may need stronger problem-solving, critical thinking, communication and decision-making skills, alongside the ability to supervise AI workflows and recognise errors. Organisations may consequently need to rethink job descriptions, recruitment criteria, qualifications, career development and training priorities.

Not every job will survive unchanged. As more tasks are automated, some roles may be redesigned, consolidated or eliminated, while new responsibilities emerge. But workforce transformation should follow an informed understanding of how work is changing, rather than an assumption that AI can simply replace entire jobs.

Why organisations may not be seeing the returns they expected

Looking at these considerations, I suspect three recurring weaknesses help explain the disappointing returns from many AI investments.

First, organisations are not sufficiently selective about what they automate. They pursue AI because the technology is available, rather than identifying tasks where it offers a meaningful advantage over existing alternatives.

Second, they fail to establish what success looks like. Without measuring current performance and defining the intended improvement, there is no objective basis for evaluating AI's contribution.

Third, they underestimate the full cost of making AI work. The impressive demonstrations rarely capture the effort required to handle exceptions, validate results, maintain integrations, adapt to changing business requirements and manage operational risks.

Even where AI performs well, the benefits may not materialise if organisations fail to redesign workflows, develop employee capabilities or translate time savings into meaningful business outcomes.

The objective is better outcomes, not more AI

AI offers tremendous opportunities to automate repetitive work, improve customer experiences, increase capacity and enable services that were previously impractical. But these opportunities need to be evaluated objectively.

At FYT Consulting, we advocate combining Human Judgement, Evidence and Automation to make better organisational decisions. The same principles should guide AI adoption.

Use human judgement to understand the business problem, identify suitable tasks and recognise the consequences of failure. Use evidence to establish current performance, test alternatives and measure the improvements AI actually delivers. Then deploy automation where the benefits justify the investment, risks and longer-term commitments.

And continue measuring performance as the business, technology and workforce evolve.

Perhaps the biggest mistake organisations can make in the age of AI is to measure success by how much AI they have deployed, rather than how much value that AI has actually created.

The objective was never to become an AI-powered organisation for its own sake. It was to become a better-performing organisation.

AI is one possible means of getting there — not the destination.