Sponsored by: Monte Carlo | Why Your Agents Keep Failing (And What to Do About It)

Summary

According to Monte Carlo's survey of hundreds of enterprise practitioners, lack of trust—primarily driven by hallucinations and performance issues—is the single biggest barrier to deploying AI agents in production, with 64% of enterprises citing incorrect or hallucinated responses as their primary concern.
This video presents a framework showing that agents fail in three places: the quality of data going in, the agent's own behavior, and the outputs that come out the other side—and that the visible failures are often just the surface of hidden upstream data problems.
A live demo traces an agent failure from a stale Delta table through a Lakeflow job to a broken vector search tool call, demonstrating how circuit breakers and automated restarts can stop failures before users encounter them.

Chapters

FAQs

Why do AI agents fail in production?

According to Monte Carlo's survey of hundreds of enterprise practitioners, agents fail for three primary reasons: agentic failures in the model's own behavior, blind spots in observability, and an overarching lack of trust. Sixty-four percent of enterprises cite hallucinated or incorrect responses as the primary trust barrier to scaling agents.

What is the iceberg problem for AI agents?

The iceberg problem refers to the fact that visible agent failures—wrong answers, slow responses—are often just the surface manifestation of hidden data quality issues upstream. In this video, a flight-rebooking agent fails because a stale Delta table is silently feeding it outdated information, a root cause not visible from the agent's output alone.

How does data observability help prevent AI agent failures?

Data observability monitors the pipelines and tables that feed agents, detecting staleness, schema drift, and quality issues before they propagate to outputs. This video demonstrates tracing a failure from a Lakeflow job through a stale Delta table to a broken vector search tool call, and using circuit breakers to automatically restart the pipeline on a data quality failure.

What did Monte Carlo's enterprise survey find about agent deployment?

Monte Carlo surveyed hundreds of engineers, AI practitioners, and leaders and found that while nearly everyone is building agents, very few are successfully scaling them in production. More than half deployed agents faster than their teams were ready to support, and more than fifty percent expect to completely rebuild their agentic infrastructure.

Full transcript

[00:08] I think we technically start in one minute, but if you've seen a talk with me, my tradition is to start with a selfie. So, you all are going to have to be in a selfie with me right now. You can say
[00:20] AI if you really want with your hands up. AI. Awesome. Thank you everyone. good participation.
[00:32] Okay, I think we can get started. Hi everyone, my name is Barb Moses. I'm the CEO and co-founder of Monte Carlo. We work with hundreds of enterprises to help them deliver trusted AI. We'll talk a little bit about what that means. We
[00:45] work with companies like Pepsi, American Airlines, Rivian um and many others around here. The title of this talk, garbage data and garbage agents. Today we're going to double click into the
[00:57] word trusted trust and how do we actually avoid delivering both garbage data and garbage agents.
[01:10] So let's start with this um this robot and this iceberg. What do these two have in common? They're both blue. Um they're both maybe cute. Debatable. And in both of them, what you don't see is often the
[01:22] problem. We're going to come back to this. to just hold this thought for a second. I want to share with you that earlier this year, we actually conducted a pretty thorough survey with many folks
[01:34] like yourself. Uh, hundreds of folks in engineering, AI, leadership, and we actually wanted to know what sort of the lay of the land. The most important takeaway that we saw is that everyone's building agent. However, where the
[01:47] blocker was that very few are actually scaling them in production. About half of those have agents in production. More than half have deployed those faster than they thought their
[02:00] teams were ready to support and about 50 more than 50% of them expect to completely rebuild their agentic infrastructure. I thought this last insight was probably the most interesting. So I wanted to double click
[02:13] here into why. And we've heard three primary reasons. The first is agentic failures. The second is blind spots and an overarching theme around lack of
[02:25] trust. So the takeaway, the key takeaway here is lack of trust is the biggest barrier to deploying agents. Now let's look into that a little bit more. What do we mean when we say lack of trust? What have folks reported?
[02:39] 64% say that incorrect or hallucinated responses limits their trust in AI. That's really the symptom, if you will. Almost half have noticed performance issues. And when your agent is slow,
[02:51] it's actually equivalent to non-existent. 59% have said that they're not fully confident that they have visibility across their data and AI estate. And about half have already discovered that
[03:05] they their agents have access data that they were not aware of. And so this last point here is super interesting. I don't know if show of if I show of hands has anyone sort of had their agent already hallucinate or produce uh wrong
[03:18] responses. About half of the audit sort of matches this. Um I don't know about you but for me you know it's sort of interesting. I found that like dashboards and data have been wrong for a very very long time but
[03:31] they're kind of they're wrong like in a subtle way. Agents and AI they're sort of wrong confidently like they argue with you. you have to be like no Gemini or no whatever it is actually I'm right
[03:43] um I find that really interesting right I think the other thing that's actually interesting in sort of the human psychology of trust for some reason when the chatbot will provide the sources even the answer even if the answer is
[03:55] completely wrong just the presence of those sources give me confidence for some reason I don't know if you all feel the same so sometimes like AI agents can be gloriously wrong and gloriously off but something in us can choose to
[04:08] believer to trust or not. Based on all of this, I want to make a claim and a thesis for the talk right now. And my thesis is that historically for the last couple of years, we've really thought of data and AI as
[04:21] completely separate systems. We've had data teams and AI teams. We've had data governance and AI governance. We've had data pipelines, data stack, and an AI stack. I'd like to make the claim that I
[04:33] actually think this is a mistake that many of us are making the shift to actually realizing the data and AI are two halves of one system. There's sort of the obvious part of this that data powers agents and agents calls on data
[04:47] to try to inform a decision to try to make a better recommendation. But the other half is less obvious. the what the agent produces whether that's a tool call a trace an eval that's data too
[04:59] that's telemetry too are we collecting that and actually observing that as well and so I think when you know folks saying that they're not actually confident about what their agents are doing I would want to make the argument
[05:12] that they're actually not looking at this full picture and when you're not looking at this full picture and we'll come back to the iceberg exam example now you're actually missing a lot of things and So what you see if it's the
[05:24] agent hallucinating or confidently giving you the wrong data sources there's actually a fair amount of things that are happening under the surface that you're not seeing especially if you're not looking at
[05:37] and this can this can look in various state can look in and um kind of come up in different ways and you know on the left hand side here you kind of see you know what you think is wrong and the left hand side what is actually wrong
[05:49] but the overarching thing here is that often times s agent failures may in fact be data failures. Sometimes agent failures are are agent failures as well.
[06:01] But there is a class of cases where it's in fact we have to lead that back to the data. And so if your team says you know the agent is just hallucinating it's not me. I can't do anything about it. Or the model is just not smart enough. We have
[06:13] to wait six months to the next version. Opus 4.8 is not cutting it. Or maybe we need to better prompts. Actually, what's happening in the background is that your iceberg table is stale or someone made a
[06:26] schema change or no one's actually looking at the code way upstream that causing all of that. So, a lot of us are focusing on the on the left hand side here and very few of us are looking at the right hand side to actually triage
[06:38] and who's caused what's going on. The bottom line here, it's very hard to prompt your way out of a stale table. And so when we're thinking about delivering trusted data and AI products,
[06:50] we have to understand both sides of this. We can't cut corners here. I want to show you where this actually gets amplified and more complicated. So an example of a rag pipeline. I won't go through all of the all of the details
[07:03] here, but if you start with the source data where you have an example a data lake table that's stale. Once you start with the chunking and the embedding, things get a lot worse. Maybe you split a data into two. Maybe the
[07:16] embeddings no longer reflect the reality of the data that you have. It gets further amplified with the vector store if your index is out of sync. And then finally, all of this at the end is uh uh
[07:30] results in an agent where the retrieval is completely off. And so even problems that start at the very very left get propagated from step to step. So catching them at the end on the right hand side is not always sufficient
[07:43] because you're not always understanding where it's propagating from. And so you know we work sort of fortunate to work with the the best data and AI teams in the world. We use AI quite extensively at Bonnie ourselves.
[07:57] And what we've learned is that the best teams and the most forward-looking organizations actually think about data and AI as one holistic system. They don't think of those as two separate things that need to be secured
[08:09] observed separately but as actually as one estate and there's sort of four key things to look at that when you cons consider that the first is the context. So when you think about an agent and that agent goes to pull data from that
[08:22] an agent in agent bricks pulls data from data bricks is it actually pulling data that's fresh on time accurate and complete that's really sort of the basic data quality data observability that we've all sort of learned and loved over
[08:35] the years it's definitely an important pet important step necessary but not sufficient we also need to start thinking about performance if I have a customer support agent and it takes it two hours to give a support uh response.
[08:50] That's not great. We want to make sure that our support agents are as good as humans or better. And so actually getting sort of alerts and being aware when the latency is off, um when an agent is too costly, you know, if one
[09:03] tool cost me 60k, I'm not going to be able to run that at scale as an example. The third is that agents make decisions and they make decisions in a certain order. So we have to understand their
[09:15] behavior. You can think of it as a decision tree. So you actually have to be able to really visualize and understand what is the set of decisions that agents made and why. I'll give you an example. If an agent first pulls data and then makes recommendations, that
[09:29] seems reasonable. What if the agent swiped the order? What if it first gave a recommendation and then went to the data to see what's going on? That probably makes less sense. For a human being, that's sort of obvious. For an
[09:42] agent, it's not wired that way. And so we want to make sure that the order of the tool calls does make sense. And then finally, the first three can be perfect and the agent can still hallucinate. It
[09:54] can still give a response that's completely not for fit. It could make a recommendation to go check out your competitor. It can give a response that's just not the type of response that you wanted. Maybe it was too long or it was too short. Maybe it just
[10:07] hallucinated something completely off topic. In these in these instances, in order for us to deliver trusted systems, we have to look at all four in a systematic way. I'm actually going to spend a
[10:19] couple minutes now on a demo to show you what this looks like in real life in production.
[10:36] Dramatic wait for the password. We'll be right back.
[10:49] >> Okay. So, I hope you can see this. I'm going to walk you through an example of an airline flight selector. So, I assume you all have come who took an air who took a flight to get here and got on a flight. Okay, large majority of you all. Um, imagine that your airline wants to
[11:02] provide you're with an agent to actually make a recommendation for what flight you need. You want to provide some sort of query like hey I need a flight to get from X from Y and you're going to expect the agent to give you the answer. What
[11:14] you see here is basically a dashboard that monitors the performance of this agent. And so what we're seeing on the left hand side is the recommendation valid validity went down from 90 99.2%
[11:26] to 94%. Now, right after that, you see that there were only 34 user complaints, but there were 2.3,000 conversations that were completely off with recommendations that were wrong.
[11:39] And all of that is from 6 hours ago. And so, actually, this is a completely silent failure, if you will. Very small number of users complaining, very few things are failing. When you actually
[11:51] look at the flight and sort of a sample trace in this instance, what you see is that someone here has asked for a direct flight. And the the issue is that this flight was actually rescheduled 6 hours ago. So the agent made a recommendation
[12:03] for a flight that was completely off. This was happening 2,000 times and we weren't even aware of it. And so in this instance, we actually have an agent that goes ahead and automatically starts to do the troubleshooting and looks into
[12:15] the root cause. It kicks off itself the troubleshooting agent and starts looking into the investigation of what's going on. It looks at the agent bricks traces. It looks through the delta table monitors and it ends up looking seeing
[12:28] that there's an upstream leg flow job that failed. It's able to say with 97.3% confidence that there's correlation between that air that upstream leg flow
[12:41] job failure and the downstream agent output. Let's double click into that to really understand what does that mean and kind of be able to visualize or root cause. What we see here is this is sort of the the lineage if you will of the impact. It starts upstream with the
[12:55] lakeflow job that failed. Continues with the D delta table that's st that's stale. It's been stale for six hours. Ends up with the vector search te tool being outdated and finally the agent
[13:09] here being degraded with giving a a high number of conversations. um uh that are not valid. And so in this instance, it's pretty clear what is sort of the glaring problem right here. But what we want to
[13:22] do if we want to double click into the lineage view starting with the agent lineage, we can actually see what you see here is basically this flight booking workflow and the various nodes, the various tool calls that the agent is
[13:35] doing. And you see that this one has been failing the check seat availability. Now, this is the agent lineage. And here's the kicker. The age in the lineage is not sufficient. We can't just stop there. What we want to
[13:48] do is cross reference this with the data lineage. And in the data lineage, what you can see here is that the agent bricks flight agent that makes these recommendations
[14:00] actually has been failing all along due to the tool call that we just saw in the agent lineage. The check seat available. That is a result of this flight inventory table that has been stale for
[14:13] six full hours. And so nothing breaks, nothing fails, but the data is late, the data is out of date, the tool call is failing, and the agent is producing the wrong uh the right answers and likely
[14:27] damaging revenue, trust um and uh productivity for uh for the customer. So what do we do about it? Okay, we troubleshooted this. Awesome. Great. What now? Um well, you'd expect to be
[14:41] able to actually drive to some resolution and that is that is really the next step. And so there's sort of a resolution playbook that's recommended here. The first and most important step is to restart the job. Pretty basic.
[14:54] Turn the switch on and off, right? You actually probably don't need a human being for that. You can probably have the agent just run that. But if you want to do this in a human-guided way, you can review these mod these recommendations and click yes to
[15:07] approve. Please restart the job. A couple other things that you might consider doing. The first is um to create a circuit breaker so that in the future if something fails upstream, the
[15:19] agent doesn't continue to make false recommendations. Right? Pretty basic, but it's not always easy to think about how to set this up. And then finally, most important, notify the data engineering in Slack. I don't
[15:32] know about you, but I don't like to hear about issues from my downstream consumers or from other people. I want to be the first to know when things fail for the things that I'm accountable for. So, actually being able to notify the people to need to know that is is quite
[15:45] critical. And then, you know, the magical thing about this, which I think is probably the coolest part, is that this entire thing actually just took 12 seconds. And I think that's pretty remarkable because when you look at the comparison, it would take a human being
[15:58] a full four hours to do that. And that's real. Who here has triage or troubleshooted an incident for for hours or days? All of us, right? And it's always at 5:00 p.m. on Friday, right before you
[16:11] want to head out. What if we actually had agents at our disposal to help us not only identify the issue which is the first most important step but also triage, troubleshoot and resolve the
[16:23] classes of cases that we can actually use agents to the full full power of those. I want I want you to sort of think about sort of one idea here and then and then we'll go back um uh we'll move on to the
[16:35] next concept. The one thing I want to think I want you to think through is like this is one agent with one incident and one table that's delayed. With all my conversations with folks today, they probably have somewhere
[16:48] between one and five agents that are going to be in production this year, but very soon folks are going to have dozens and hundreds of agents. I don't think I spoke to a single person here who didn't see lots of agents in their future. I
[17:01] think the only way for us to move at agent speed is to also think about agent observability and agent trust because that's how we're going to be able to deliver these with the right intentionality. We can actually move at that pace, but we're not going to be
[17:14] able to manually look at these. What folks are doing today oftent times is the manual uh part of it. So let's talk about what are the takeaways that you all can leave with today.
[17:33] complicated tech, but I think we're going to get there. Oh, whoops. Sorry, guys. These are screenshots. Wh There you go. Okay. I'd like to leave you with something practical for today, which is a
[17:45] scorecard that you can take back to your office on Monday morning. Maybe take Friday off. Um, and actually score yourself. have an understanding of where you are today and choose where do you
[17:57] want to be in 30 days, in 90 days. Make one decision. Let me just walk you through a little bit of what this is. The top there's sort of three different levels. First level, blind. That means you're not in control. You hear about
[18:10] issues from others. You learn about it from your stakeholders, from your customers, from that angry user who didn't get the right flight recommendation. The second level is reactive. You probably have something but it's very
[18:23] manual. You learn about an issue after it happened. You don't have anything proactive. And the third level is production grade and that means agentic first monitoring, troubleshooting and remediation.
[18:37] Now let's talk about the different layers. The first is the data layer. Most folk here are somewhere between blind and reactive. Many I would bet here are blind. On the second, the semantic layer. That's typically a one-time check that
[18:50] folks do. and then kind of forget it. And then the third layer, the agent bill layer, I would say most f folks are are in reactive right now. They might have a couple of eval setup or something like that. And when you think about sort of the fourth layer, that really means sort
[19:04] of having a gentic level um uh view of this whole thing. And so um again, to the degree that this is helpful, I encourage you to actually score yourself. go back to the office, go back
[19:16] to your team members, ask yourself where are we against this on each of the layers and where do we want to be and take concrete next steps to there. Um, I'll leave you with this one closing thought. The future of observability is
[19:31] fully autonomous from data to agent and the main question is are you ready for it? So if you'd like to learn more on how to do that, you can visit us at booth 206 to see autonomous
[19:43] observability from data to agent in action. Thank you so much for the time. Really appreciate it.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.