The Analytics Idea Sounds Good. But Should We Build It?
Imagine an analytics proposal landing on your desk for approval.
The idea sounds good. Perhaps the team wants to build a model to predict which customers are likely to leave, create a predictive-maintenance solution for expensive equipment, or develop an AI assistant to help employees find information more quickly. The technology appears capable, the team is enthusiastic, and someone may already have put together an impressive demonstration.
Now the proposal is asking for something more substantial: six months of work, a project team, perhaps an external vendor, integration with existing systems and a meaningful budget.
Would you approve it?
Most of us would probably want to know more. Not because the idea is bad, but because an interesting idea and an investment decision are two different things. Before committing money, people and time, we would want some evidence that there is a worthwhile problem to solve, that somebody will be able to use the outcome, that suitable data exists, that the potential value justifies the effort, and that the solution has a realistic chance of working in the organisation.
Yet with analytics, and increasingly with AI, we sometimes move surprisingly quickly from “That sounds useful” to “Let's build it.”
Perhaps there is a conversation missing between the two.
Before we build, what should we know?
This question is at the heart of a recent study by Sebastian Junker and Isabell Weinmann on how organisations manage Data & Analytics use cases. Working with 17 experienced analytics leaders through three iterative consultation cycles, and subsequently examining their framework against three industry cases, the researchers identified an interesting weakness in the way analytics projects are commonly managed. Considerable attention tends to go into developing solutions, while the earlier work of generating, assessing and prioritising worthwhile use cases receives much less attention. Existing approaches also provide limited support for deciding when a use case should progress or be terminated.
Their proposed framework contains six stages:
Ideate → Pre-assess → Assess → Prototype → Develop → Operate.
But what interests me most is not the six-stage process itself. It is the thinking behind the transitions. Moving forward requires evidence. At various points, somebody needs to decide whether what has been learned so far justifies committing further resources.
Seen from a manager's perspective, that is really an investment question. If I were being asked to approve an analytics project, I would first want to understand the problem. Is something genuinely happening in the business that is important enough to solve? This sounds obvious, but new technology can easily reverse the conversation. Instead of starting with “We have this problem. Could analytics help?”, we start with “We now have generative AI. What can we do with it?”
There is nothing wrong with experimenting with new technology. Some valuable innovations will begin exactly that way. But before experimentation becomes substantial investment, we should be able to explain what problem or opportunity makes the investment worthwhile. Junker and Weinmann's practitioners strongly emphasised problem-oriented, bottom-up ideation, with one describing ideas arising from real departmental problems as the strongest source of valuable ideas. They also warned about unrealistic expectations of what analytics and AI can do.
I would then want to know what happens if the analytics actually works. Who will use the result? What decision will improve? What will somebody be able to do differently?
This is particularly important from an FYT perspective because we often describe analytics as a journey from Data → Insights → Decisions → Action. The destination is not the dashboard, model or AI application. The destination is a better decision or action.
Imagine that we build a highly accurate dashboard identifying customers who are likely to leave. It performs beautifully. But the customer-service team has no authority to offer a retention incentive, no budget for intervention and insufficient resources to contact the customers identified.
The analytics may work. But what business problem have we actually solved?
The same applies to the data. An organisation considering predictive maintenance may have millions of sensor readings and years of operating records. At first glance, it looks data-rich. But perhaps equipment failures are extremely rare, failure events were never recorded consistently, or the maintenance records cannot reliably be linked to the sensor data. Suddenly, having lots of data is not the same as having the data needed to answer the question.
This is one of the areas where Junker and Weinmann's research is particularly useful. Their pre-assessment stage explicitly calls for early screening of data availability, accessibility and signal quality. Their argument is that, unlike many conventional IT projects, the feasibility of an analytics use case often cannot be judged from requirements and architecture alone. We need some empirical indication that the necessary data and signal actually exist.
Then there is value. Even if the problem is genuine, the analytics works and the data supports it, how much difference could the solution realistically make? Value might mean increased revenue, lower costs, faster work, better decisions, reduced risk or an improved customer experience. The important point is that there should be a credible connection between the analytics and an outcome the organisation cares about. Junker and Weinmann note that technical model performance and business value do not necessarily move together; a technically successful model can still deliver disappointing business value.
Finally, the solution has to survive contact with the real organisation. Will people trust it? Will they use it? Does it fit their workflow? Who owns the resulting action? What changes will be required? Are there privacy, security, compliance, explainability or other risks that materially affect the case for investment?
The researchers' assessment stage explicitly includes these types of risks alongside deeper examination of data readiness, the business case, resources and implementation.
Taken together, I find it useful to translate these considerations into five practical lenses for an investment conversation:
Problem → Decision → Data → Value → Adoption & Risk
This is not Junker and Weinmann's framework. It is an FYT practical lens informed by their research and by the way we think about analytics as a business decision rather than simply a technical exercise.
Three good ideas. Three reasons to pause.
These considerations become clearer when we apply them to apparently attractive projects.
Take our customer-churn dashboard. The business problem is real, the data may be good and the prediction may be accurate. But if nobody has the authority or resources to intervene, the weakness is not really analytical. We have failed to establish how the insight will lead to a different decision or action.
The predictive-maintenance project has a different problem. Equipment failure may be extremely expensive, so the potential value is obvious and the organisation has every reason to want a solution. But if the historical failure data is too sparse or unreliable to identify a useful pattern, enthusiasm and business value cannot manufacture a signal that is not present in the data.
Then consider the AI assistant that management is keen to build because the organisation wants to “do something with AI.” A prototype may be remarkably easy to create. It might answer questions and produce an impressive demonstration within days. But if nobody has clearly identified the user problem, the workflow it will improve or the value it will create, technical feasibility tells us surprisingly little about whether it deserves a major investment.
These projects have not necessarily failed. In fact, none of them may need to be abandoned. What they have done is reveal a question that needs answering before the next level of commitment.
What if the honest answer is: we don't know yet?
This is where I would be careful not to turn assessment into bureaucracy.
If the predictive-maintenance team cannot yet tell whether there is enough signal in the data, should management approve the entire project or reject it? Neither response may make much sense. Perhaps the right decision is to give two analysts a few days to explore the data and find out.
Likewise, if we are unsure whether employees would find an AI assistant useful, perhaps we should test a limited version with a small group before committing to enterprise-wide deployment. The purpose of that experiment is not merely to produce an impressive demo. It is to buy information that helps us make a better investment decision.
This distinction is important because analytics contains uncertainty. Sometimes we cannot know whether an idea will work until we analyse the data, test an assumption or put a prototype in front of real users. The aim should therefore not be to prove that a project will succeed before spending anything. If we demand that level of certainty, we will probably prevent worthwhile experimentation.
Instead, I would use a much simpler principle: The evidence required should be proportionate to the commitment being requested.
A few days of analyst time to explore whether a useful signal exists does not require a 30-page business case. A small prototype involving a handful of users should require more justification, but still leaves room for uncertainty. A six-month development involving external vendors, system integration and substantial organisational change deserves a considerably stronger case.
This is also why the distinction between a prototype and a project matters. A working prototype tells us something, but not necessarily everything we need to know. A prototype AI assistant may show that the technology can answer questions from company documents. It does not automatically tell us that employees will use it, that its answers are sufficiently reliable, that access to sensitive information can be controlled, or that the value created will justify the cost of deploying and maintaining it.
There is a broader lesson here. Analytics and AI create value not simply because an organisation possesses the technology, but because that capability ultimately changes decisions, processes or outcomes. That is consistent with wider research examining how analytics capabilities contribute to innovation and business performance rather than treating the technology itself as the end result.
So perhaps the question at the early stage should not be: “Can you prove this will work?”
It should be: “What is the smallest sensible investment that will give us the evidence we need for the next decision?”
And sometimes the evidence says stop
This is probably the hardest part.
Once an analytics project has started, our relationship with it changes. People have invested time. Money has been spent. A senior executive may have championed the project. Teams have made commitments and reputations can become attached to its success. At that point, stopping feels very different from deciding not to start.
We begin hearing things such as, “We've already spent so much on this,” or, “Let's give it another three months.” But money already spent does not make the next dollar a good investment.
Junker and Weinmann identified this as an important weakness in existing approaches to analytics use-case management: criteria for progressing or terminating projects are often not sufficiently operationalised. Their stage-gate approach deliberately creates points at which evidence can be considered and a go, no-go or hold decision made. It also recognises that analytics development itself can expose new uncertainties around data, model behaviour, adoption and implementation.
For managers, I think we can translate those decisions into four perfectly legitimate outcomes: Proceed | Revise | Hold | Stop
Proceed when what we have learned justifies the next investment. Revise when the underlying problem remains worthwhile but the proposed approach needs to change. Hold when the idea still makes sense but an important dependency is not ready. Stop when the evidence tells us that further investment is no longer justified.
Stopping, in particular, should not automatically be regarded as failure. If a relatively inexpensive assessment reveals that the necessary data does not exist, that users cannot act on the result or that the likely benefit is too small to justify the cost, the assessment has done something valuable.
It may have prevented a much more expensive failure.
Sometimes the most valuable outcome of an analytics assessment is deciding not to build the analytics.
Approval is not forever
There is one final complication. Suppose the project passes the assessment. The problem is important, the data looks promising, users are interested and management approves the investment.
Does that settle the matter?
Not necessarily, because analytics projects have an awkward characteristic: uncertainty does not always decline neatly as development progresses. New uncertainties can emerge when we work with the data, test models and introduce the solution into the organisation. Junker and Weinmann specifically note that data dependencies, non-deterministic model behaviour, user adoption and a changing regulatory environment complicate analytics use cases in ways that conventional project approaches may not fully accommodate.
Imagine that our churn project looked excellent when it was approved. Three months later, however, the team discovers something unexpected. The customers the model can identify most accurately as likely to leave are also the customers least responsive to the interventions the organisation can realistically offer.
The model may be working.
But the business case has changed. Would we continue simply because the project was approved three months ago?
This is where the stage-gate thinking becomes particularly useful. The original approval should not give a project a permanent licence to consume resources. New evidence should be allowed to strengthen the case for continuing, change the direction of the project or, when necessary, weaken the case enough that we stop.
An analytics project should keep earning the right to continue.
Where does this fit with the actual analytics?
This thinking does not replace FYT's existing Analytics Thought Process:
Define → Hypothesis → Prepare → Analyse/Test → Interpret → Communicate
The two are answering different questions.
Our Analytics Thought Process helps us think rigorously about how to analyse a business problem. We define what we are trying to understand, consider possible explanations, prepare the necessary data, analyse and test, interpret what we find and communicate it so that someone can make a better decision.
The question in this article sits one level earlier and, in some ways, around that process. Should this proposed analytics use case receive the investment needed to become a project in the first place? And as we learn more, does it still deserve further investment?
That also makes this question different from one I explored previously in the FYT Question Library: The Dashboard Looks Good. But Is It Answering the Right Question? That article was concerned with whether we had properly defined the business question before analysing the data. Here, we may have a perfectly legitimate business problem and still decide that a particular analytics solution should not proceed.
Perhaps the data is inadequate. Perhaps nobody can act on the result. Perhaps the expected benefit is too small. Perhaps the risks are too high. Or perhaps we simply do not know enough yet and should invest a little in finding out before investing a lot in building.
So the next time an analytics or AI idea sounds exciting, perhaps we should resist moving immediately from “That could be useful” to “Let's build it.” A better conversation is about what we need to know before making the next commitment.
We do not need certainty before we begin. Analytics will always involve exploration, experimentation and learning. But as the money, people and organisational commitment increase, the strength of the evidence supporting that investment should increase with them. And even after approval, new evidence should be allowed to change the decision.
A good analytics idea still has to earn the right to become a project. And once it does, it should keep earning the right to continue.
Mini-Appendix: Quick Reference
Analytics use case
A specific application of data or analytics intended to improve a business decision, process or outcome. Junker and Weinmann use the term broadly enough to include traditional analytics, advanced analytics and generative AI.
Pre-assessment
An early, pragmatic evaluation before notable resources are committed. In Junker and Weinmann's framework, this includes initial technical and business feasibility, portfolio prioritisation and early screening of data availability, accessibility and signal quality.
Data readiness
Whether the necessary data exists, can be accessed and is sufficiently suitable and reliable for the proposed analysis.
Signal
A meaningful pattern or relationship in the data that can support the analysis or prediction we want to make. Large quantities of data do not necessarily mean the required signal exists.
Prototype
An early version of a solution used to test important assumptions and reduce uncertainty before committing to full development.
Sunk cost
Money, time or other resources already spent that cannot be recovered. Those past costs should not, by themselves, determine whether further investment is worthwhile.
Stage gate
An approach in which work progresses through stages of activity and evidence gathering, with explicit decision points between them. Junker and Weinmann adapt this logic specifically to the characteristics of Data & Analytics use cases.
Research that informed this article
The main inspiration for this article is Sebastian Junker and Isabell Weinmann's “Managing Data & Analytics Use Cases,” published in the Schmalenbach Journal of Business Research on 23 September 2026. Their proposed D&A stage-gate process was developed through three iterative cycles with 17 experienced industry practitioners and assessed retrospectively against three industry cases. The authors position their resulting constructs as an intermediate theory that should be explored and tested further rather than as a universally validated formula.
Comments