Webinar: Why Most Gen AI Pilots Fail and How to Fix Them

Why Most Gen AI Pilots Fail and What to Do About It

Generative AI is everywhere right now.

Leadership teams are talking about it. Departments are testing it. Employees are using tools like ChatGPT, Copilot, and Gemini to speed up everyday work. For many organizations, the pressure is clear: “We need to do something with AI.”

But that pressure is also where many Gen AI pilots start to go wrong.

In Cadeon’s webinar, Why Most Gen AI Pilots Fail and What to Do About It, Phil Ungar, Greg Kreutzer, and Jason Jenkins explored why so many AI initiatives fail to move beyond experimentation. The session focused less on the hype around AI and more on the practical issues that determine whether a pilot becomes a real business asset or just another abandoned project.

The biggest takeaway was simple: most Gen AI pilots do not fail because the model is not powerful enough. They fail because the organization is not ready.

AI Is Not the Starting Point

A lot of organizations begin with technology.

They see what generative AI can do and immediately start asking where it can be applied. That is understandable, but it can also be risky. Starting with AI before understanding the business problem, data environment, workflows, and governance needs can create a fast-moving mess.

Phil described this as the danger of doing “AI first” instead of “data first.” The point is not that AI is bad. The point is that AI depends on the information, processes, and people around it.

If the data is scattered, the process is unclear, and the business outcome is vague, even a strong AI tool can produce weak results.

This is especially true now that organizations are working with more data than ever. Every system, device, vehicle, application, and customer interaction can create data. But more data does not automatically mean better intelligence. The value comes from knowing what data matters, where it lives, how it is structured, and how it should be used.

Not All AI Is the Same

One of the first challenges with AI projects is language.

People often use the term “AI” to mean very different things. A board member may be thinking about ChatGPT. An operations manager may be thinking about predictive maintenance. A technical team may be thinking about AI agents that can take action across systems.

All of these can be valid AI use cases, but they are not the same.

Jason made an important point in the webinar: it is more useful to talk about what the AI is supposed to do than to get stuck in technical labels.

• Is the goal to predict future outcomes?
• Is the goal to generate content or summarize information?
• Is the goal to automate a multi-step process?
• Is the goal to support better decision-making?

That functional clarity matters because each type of AI requires different data, controls, risks, and success measures.

Predictive AI depends heavily on clean historical data. Generative AI depends on the quality and structure of the content it is grounded in. AI agents raise deeper governance questions because they may take actions across systems with different levels of human approval.

The more powerful the AI use case, the more important readiness becomes.

The Real Reasons Gen AI Pilots Fail

The webinar identified four common areas where Gen AI pilots break down:

• Problem clarity
• Data readiness
• System readiness
• Change readiness

These are not mainly technology problems. They are business and operating problems.

That is why buying another tool rarely fixes the issue.

1. The Business Problem Is Not Clear Enough

Many organizations start an AI pilot because they want to “use AI.” But that is not a business objective.

A better starting point is asking what specific problem the organization wants to solve.

• Do you want to reduce manual reporting time?
• Improve response times?
• Summarize large volumes of documents?
• Support field teams?
• Improve internal knowledge search?
• Reduce repetitive administrative work?

If the problem is not clear, the success criteria will not be clear either.

That is one of the most common reasons AI pilots lose momentum. Teams may say they want “efficiency,” but they have not defined what efficiency means, how it will be measured, or what baseline they are starting from. Without that, it becomes difficult to prove whether the pilot worked.

A strong AI pilot should be tied to a measurable business outcome from the beginning.

Not “Can we use AI here?”
But “Can AI help us improve this specific workflow, metric, or decision?”

2. The Data Is Not Ready

Data readiness is one of the biggest issues in Gen AI projects.

Many companies technically have the data they need, but it is not in a usable state. It may live across different systems, folders, formats, PDFs, spreadsheets, scanned documents, old reports, or disconnected applications.

That becomes a problem when AI is expected to produce accurate answers.

Generative AI can sound confident even when it is wrong. If the system is pointed at messy, incomplete, poorly labelled, or unstructured information, it may fill in the gaps with incorrect answers. These are often called hallucinations.

The fix is not always a better AI model. In many cases, the fix is a better data foundation.

That does not mean every organization needs perfect data before starting. Waiting for perfect data can delay progress indefinitely. But organizations do need enough structure for AI to work reliably.

That might include:

• Clear file naming
• Organized folder structures
• Useful metadata
• Defined document types
• Known data sources
• Clear access rules
• A basic understanding of what information the AI should and should not use

Those steps may not sound exciting, but they can dramatically improve the relevance and reliability of AI outputs.

3. Governance Comes Too Late

Many organizations only start asking governance questions near the end of a pilot.

That is a problem.

By the time a solution is ready to launch, it may already be difficult or expensive to adjust how it handles data, permissions, accountability, and risk.

The webinar raised several questions organizations should answer earlier:

• What data can the model access?
• Can it use external internet data?
• Who is accountable if an AI-assisted decision is wrong?
• What guardrails control the output?
• How will hallucinations be handled?
• When does a human need to stay in the loop?

These questions do not always fit neatly inside existing IT or data security policies. AI creates new risks, especially when systems generate answers, influence decisions, or act across tools. Governance needs to be designed before deployment, not after something goes wrong.

Good governance should not be about slowing everything down. It should create enough structure for teams to move safely and confidently.

4. People Are Not Ready to Use It

Even a well-built AI solution can fail if people do not adopt it.

This is one of the most overlooked parts of AI implementation. Many organizations invest heavily in the tool but not enough in the people who need to use it.

Employees need to understand what the AI tool does, where it fits into their workflow, what they can trust, what they should verify, and how it changes the way they work.

If people are not trained, they may avoid the tool entirely. Or they may use it in ways that create risk.

That is why change readiness matters. Adoption needs to be planned before launch, not treated as a final training session at the end.

People, Process, and Data Matter More Than the Tool

One of the strongest ideas from the webinar was the shift from “people, process, and technology” to “people, process, and data.”

Technology still matters, but it is rarely the only issue.

There are already many strong AI tools in the market. The harder question is whether the organization has the right foundation to use those tools well.

• Are leaders aligned?
• Is the business problem clearly defined?
• Is the process understood?
• Is the data usable?
• Are the right controls in place?
• Do people know how to use the solution confidently?

Those are the questions that separate a successful AI pilot from one that stalls.

Automating a Bad Process Just Makes the Problem Move Faster

A particularly useful moment in the webinar came during the discussion about process.

Phil pointed out that many Gen AI projects are really about automating part of a workflow. But if the organization does not understand that workflow end to end, automation can simply push the bottleneck somewhere else.

Greg summarized it well: if your processes are not modelled properly, AI may help you do the wrong things faster.

That is a critical point.

AI should not be layered onto a broken process without first understanding how the process works, where it creates value, and where the friction really happens.

Before automating, organizations should ask:

• What is the full process from start to finish?
• Where does the delay happen?
• Which steps require judgement?
• Which steps are repetitive?
• Which parts should stay human-led?
• What happens downstream if this step becomes faster?

Without that clarity, AI can create speed without improvement.

What Successful Organizations Do Differently

Successful AI teams are not always the ones that move fastest.

They are the ones that prepare better.

• They define the business problem before selecting the tool.
• They understand the data before expecting useful outputs.
• They build governance before deployment.
• They prepare people before launch.
• They create a roadmap that fits the organization’s reality.

This does not mean slowing down forever. It means doing enough readiness work to avoid wasting time, money, and trust later.

The organizations that get the most value from AI are usually the ones that build the foundation first.

Cadeon’s AI Best Practices Framework

To help organizations assess readiness, Cadeon uses an AI Best Practices Framework built around five key areas:

• AI strategy and leadership
• Data infrastructure
• Architecture and systems
• Governance and risk
• Process and change management

Each area connects back to people, process, and data.

The purpose of the framework is not to create theory. It is to help organizations understand where they are ready, where they are exposed, and what needs to happen before they move forward.

The real question is not simply, “Are we ready for AI?”

A better question is:

Where exactly are we not ready, and who needs to do what?

That level of specificity is what helps AI initiatives move from vague ambition to practical action.

What the AI Assessment Provides

Cadeon’s AI Best Practices Assessment is designed to give organizations a clear, practical view of their current readiness.

It looks at where the organization stands today, compares that against best practices, identifies the most important gaps, and creates a roadmap for next steps.

The assessment is built around four outputs:

• A current state assessment
• A best practice benchmark
• A prioritized gap analysis
• A right-sized roadmap

The roadmap is especially important because every organization moves at a different pace. Some may need a focused 90-day action plan. Others, especially in government or regulated industries, may need a more detailed roadmap that accounts for approvals, communication, risk, and internal change management.

The goal is to help leaders make a confident decision about their Gen AI pilot or broader AI program.

Final Thoughts

Generative AI can create real business value, but not when it is treated as a shortcut.

The organizations that succeed with AI are the ones that understand what they are trying to improve, prepare the data behind the system, align the people who will use it, and design governance before the risks appear.

That is the real lesson from the webinar.

AI pilots do not usually fail because the model is not advanced enough. They fail because the business foundation around the model is not ready.

Before launching another Gen AI pilot, organizations should pause and ask:

• What problem are we solving?
• What does success look like?
• Is our data ready enough?
• Do we understand the process we want to improve?
• Who owns the outcome?
• What guardrails do we need?
• How will people actually use this?

Answering those questions early can save a lot of time later.

AI is moving quickly, but speed alone is not the advantage. The advantage belongs to organizations that build the right foundation, choose the right use case, and turn AI from an experiment into a practical part of how the business works.