Self-Improving Agents and Agent Evaluation With Arize & Databricks ML Flow

Summary

Self-Improving Agents: Build resilient, autonomous agents that continuously self-improve through iterating feedback loops, essential as agents become more sophisticated and widely deployed.
Agent Evaluation Techniques: Combine tools from Arize, Databricks MLflow, and Mosaic AI to evaluate and improve agent performance, addressing risks from malicious inputs and undesirable outputs.
Production Resilience: Draw from real-world productionized use cases to implement practical strategies for building and maintaining high-performing agents on the Databricks Data and AI platform.

Self-Improving Agents and Agent Evaluation With Arize & Databricks ML Flow

Talk by Aparna Dhinakaran, Co-Founder and Chief Product Officer, Arize
As autonomous agents become increasingly sophisticated and widely deployed, the ability for these agents to evaluate their own performance and continuously self-improve is essential. However, the growing complexity of these agents amplifies potential risks, including exposure to malicious inputs and generation of undesirable outputs. In this talk, we'll explore how to build resilient, self-improving agents. To drive self-improvement effectively, both the agent and the evaluation techniques must simultaneously improve with a continuously iterating feedback loop. Drawing from extensive real-world experiences across numerous productionized use cases, we will demonstrate practical strategies for combining tools from Arize, Databricks MLflow and Mosaic AI to evaluate and improve high-performing agents.
Databricks Named a Leader in the 2025 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms: https://www.databricks.com/blog/databricks-named-leader-2025-gartner-magic-quadrant-data-science-and-machine-learning
Build and deploy quality AI agent systems: https://www.databricks.com/product/artificial-intelligence
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

Chapters

FAQs

Why do autonomous agents need to self-improve?

According to this video, as autonomous agents become increasingly sophisticated and widely deployed, the ability for agents to evaluate their own performance and continuously self-improve is essential for maintaining quality and managing risks.

How does the feedback loop drive self-improvement?

This video explains that both the agent and evaluation techniques must simultaneously improve with a continuously iterating feedback loop to effectively drive self-improvement in production systems.

What are the key risks when deploying agents?

The video highlights that growing agent complexity amplifies potential risks, including exposure to malicious inputs and generation of undesirable outputs, making robust evaluation essential.

How do I combine Arize, MLflow, and Mosaic AI for agent evaluation?

This video demonstrates practical strategies for combining tools from Arize, Databricks MLflow, and Mosaic AI to evaluate and improve agent performance, drawing from real-world productionized use cases.

Full transcript

`0:00` Introduction to agent challenges

Good morning. Good morning everybody. Hope you all had your coffee and uh wide awake. So I will take this opportunity to introduce uh Aperna Dinakaran uh who is the f co-founder of Arise and also comes with the background from Uber I believe right so well it's she's all yours but one thing she wants this to be interactive anytime you have questions please walk over to your uh to the mic there and ask questions she doesn't mind being disturbed when she's presenting a content All good.
Okay. Have a great morning. I'm going to talk about self-improving agents and agent evaluation here today. We're going to actually go through a little example of how to build this using Mosaic ML. We're going to talk about how to set up evaluations. And I think for many of you in this room, there's like I I think there's a around building the agents. What we're seeing with a lot of teams is that evaluation and reliability continues to be one of the main challenges.
And so I'm going to walk through some of the core ways that teams will actually we're seeing teams actually evaluate agents in the real world today. A little intro about me before I hop in. I'm one of the founders of Arisei. We focus on LLM evaluation across the entire life cycle. So all the way from development into production. Um was an engineer for many years prior to this. Uh and like he mentioned worked at Uber actually bu building a big part of Michelangelo.
So with that I'll get started. Um for many teams that we actually see building these agents this is actually what their day-to-day workflow looks like where they're sitting in an Excel sheet. somebody, it could be the AI engineer, it could be technical persona, uh it could be a product manager, they're sitting in there and they're actually iterating on their prompts in an Excel sheet. Um, raise your hands if you've seen this.

`1:57` Agent development pain points

You don't need to out yourself, but um this is actually what for a lot of teams iterating on their agent looks like. And by the way, this is just one prompt in often a series of um series of prompts or a series of tool calls that they're actually iterating on. What makes this really hard is that it's pretty hard to actually iterate and develop pretty quickly when you have no sustainable way of actually doing uh evaluation of your experiments.
And so one of the things that we see teams really struggling with is how do they make sure that every time they change a prompt, they change a model, they modify the tool call definition, that they actually know that it's making a pretty decent impact on the outcome of their model. Is it only fixing some edge cases and messing up other cases that they should be worried about or is it actually overall improving the performance of their agent?
And this continues actually when they go into production as well. I'm actually going to go through and give you a demo of our own agent, but it's pretty hard. In in our agent, we actually have five or six sub agents that our co-pilot actually calls. And these agents actually have tools. And so, it's actually pretty hard to understand within our own agent, what are the agents that are actually the bottlenecks in the application?
Where are the tool call definitions, let's say, the weakest? Where should we actually go fix? And so there's a pain when you're actually building the agent. There's a pain when you're actually shipping and deploying the agent into production to understand and get visibility into these these agents. And so today what I'm going to go through and I'll jump into a demo really quickly because it's always easier to see this stuff.
But let me first start off with there's kind of four main things that we think about when we talk about agent evaluations. And I'm going to dive into each one of these. is we'll do a little conceptual overview and then we'll go into a demo so we can actually see what these actually look like. So the first one's actually going to be evaluating tool calls. Most of these agents typically have a variety of number of different like this is tool call A, tool call B, tool call C that these agents actually have access to and each of these tool calls actually have arguments that need to be passed in correctly in order to properly call the tool for the agent. every time it's asked a question or there's some interaction, it actually has to make a decision of what's the right tool call to call and then what are the right arguments from the conversation or the context to correctly pass into the tool call.

`4:01` Key types of agent evaluations

And so this is kind of one of the first decisions that an agent actually has has to make. Typically in some of these more long-ending types of conversations, um you often have to call the tool call in a specific order or in the specific trajectory. And sometimes what we see these agents do is that it might take it typically four or five steps to answer a question, but it might end up taking an extra six or seven. And so you actually really want to make sure that the agent eventually converges to a pretty consistent set of tool calls so that it can actually make the right trajectory every time.
So these are what we call trajectory evals for the agent. The next one we're going to talk a little bit about is called session evals. These are not usually single turn conversations with an agent. These are typically a back and forth between a human and an agent. And sometimes you need the whole context of that session to actually understand is the agent doing what you expect it to do. So some of the questions that teams ask is things like uh is the tone consistent across the entire conversation?
Um is it actually using the context from the n minus one previous steps in the conversation to actually answer the nth next step in in that conversation? Um does it repeat questions? So which means it doesn't have state or memory actually tracking within that agent. So this is kind of one as you get into more multi-turn interactions we actually see teams go into and start to evaluate. And then this is probably one of the more complex ones which we'll get into.
But so much of the attention of what we talk about when we think about agent evaluation is actually the graph on the left here which is all about improving the actual agent. um which is really important and we the way we see teams do this is actually they collect data into production they run evals um they're consistently collecting data sets of failures they're using that to then go back iterate on their prompt and then continue this cycle but a really important part of this is actually how you evaluate your agent and most teams actually leave this as a pretty static evaluation they write the eval template once and then they forget it but typically ally these eval templates are actually the most important piece to knowing if your agent's actually performing correctly.
And so what we actually see a lot of teams do is consistently have a loop to improve how you evaluate your application so that doesn't stay static. And the way the teams do this is typically they will uh they will collect data sets of the failures of the way that the eval gradate those iterate on the eval template and use that to really kind of improve the agent while they're alongside improving the agent evaluation.
Um I'm going to go into a demo now so you can actually start to see all of this come into come into um fruition. So, we're going to go through a little bit today of if you're building an agent using Mosaic ML, how to integrate and set up evals. Um, and then evaluate that agent actually in production. And we'll go through kind of those three or four different types of evals that we talked about um tool calling evals, trajectory eval, session evals, and then how do you create a loop to do these type of self-improving evals.

`7:56` Mosaic ML implementation

So, first um if you're doing this workflow using using Mosaic, Mosaic has a number of different ways to basically um I think we're going to go through a workflow where you're calling a lane graph agent. You're going to set up tools for that agent and then you're going to actually set up evals for this agent. Um so, let me hop in and go into the demo over here. Okay, awesome. Let's uh let's jump in uh because that was that was enough time.
Okay. So, awesome. So, what I'm going to go through right now is basically a workflow where I'm going to show you how to set this up using this is an actual tutorial using the Mosaic AI agent framework. I think most of this I'm sure if you guys are using Mosaic ML, you've actually probably done. But typically, the way that most teams will set this up is using a instrument. So, you'll go through, you'll set up on Mosaic how you want to actually define your agent.
And what you're seeing here is actually just the tracing setup. If you're building any type of agent, I can't stress enough how important tracing is to actually see all the different steps and spans that your agent actually calls. So in our case, what we do is we have a pretty simple integration no matter what framework you're actually building with. In this case, it's using the lang chain instrument. But if you're building with any framework, so if you're using llama index, autogenai, um all of these frameworks are actually just a single call, single line instrumenter, you can set this up and then once you actually set up the inventor, it actually instruments your entire application.
So in this simple example, this one's actually calling um a code kind of building agent. There's a couple different tools here that it's actually using to actually call this this entire agent once you actually set up that instrument that I showed you. What you can do is actually see the entire trace of your agent over here. So in this case, this is again my lane graph agent. Let me zoom in here. You can actually see in this case uh the agents actually has access to a couple different tools.
So it's actually calling those tools inside and then it's actually calling the model to then go and write write some code. In this case I think it's just a simple math operation. Um the eval that we've run on top of this one is eval to do simple kind of code readability or code functionality. So does it have bugs? Is it actually readable? And teams actually use this to typically set up full tracing of their application, view what's happening, and then actually set up eval to be able to debug and evaluate what step in their application this this is going wrong.

`10:39` Analyzing tool and trace performance

So just wanted to show you guys how you can set this up using Mosaic. What I'm actually going to go through today is actually our own agent just so you guys can see it. Um this is a kind of visualization of our own agent just for teams who um who don't know about it. Uh our copilot basically has think about it more like an orchestration worker type approach where we have a planner. A planner actually decides what route that this agent should actually go through.
The planner as you can see here I'm actually evaluating all the different paths that the planner can go through. Um and then the planner actually decides in this case what tool calls to actually route to and and call for this agent. So in this case depending on what the input is um we have all sorts of different actions that this co-pilot can call. It can call something like a search and you can actually use our co-pilot to ask questions inside the product.
You can actually call our co-pilot to um evaluate and improve prompts. And so that's a different tool that you can actually go through and it can actually run evals for you. And so depending on the input that the user puts in, we actually decide what tool call to actually go into. So let me actually show you a simple trace of this. Um in this example, the user actually asked a simple question um being able to let me just show you what that trace looks like.
Users actually asked us a question like find me incorrect ingestion questions. In this case, what we've done is actually called a tool call. So, it's actually routed to the search tool call the tool function. You can actually see all the different functions that this tool call actually has. This over here is actually the first of the eval right now, which is tool calling evals. So, in this case, our agent actually had four different tool calls that it could have called.
One, when a user asks a question to only look across a single column and answer the question. One, to actually look across the entire table. So this is kind of a table search to look across multiple columns and answer the question. If the user is asking a more simple, hey, map this text to a filter. This is kind of what that tool call does. And then this tool call actually does kind of light analysis around what the user is actually asking.
And so the first step that this agent actually has to do is decide based off of what the user's question was, which in this case was find incorrect injection questions. It actually has to go in and decide which of the actual functions to call and has to make the right selection and then pass in the right arguments into the tool call itself. So in this case, our tool call had three different input arguments. it has to pass the context from the question that the user has and answer that you know and call the tool call itself and so we actually have eval that are running on both of those things.

`14:18` Advanced evaluation metrics

First is actually correctness of whether it's actually calling that tool call itself and then the second one is actually argument correctness whether it's actually passing the right arguments into the tool call. So these actually help you evaluate the first of the four that we were kind of talking about earlier which is tool call incorrectness and it's helpful to have these binary labels correct incorrect but actually what we found most helpful is actually the explanations because those actually give you direction of hey in this case it found the right tool call to call.
So it's calling the search tool call. It's it's calling it on a single column, but what it's not doing right is it's not passing the right arguments into the tool call. So you actually can read the explanation and see well because the search columns invalid, the overall arguments are deemed incorrect. Therefore, the arguments don't line with the required parameters leading to the conclusion that the arguments are incorrect.
So now what I can do is actually when I'm building my agent start to figure out well if is it actually passing the right argument and if it's not passing the right argument I can decide to modify that or go back and iterate on the actual tool calling definition to make sure that it pulls the right arguments into the call um to define it properly. So this is the first one that we went through. Typically what teams will do is once you identify these types of you know where the there's bottlenecks in the application what teams can actually do is pull these into our playground.
So you can actually load all of this into the playground and we have teams actually side by side compare multiple prompts against each other. So I have version one compared against version two and they'll actually test these on the outputs to make sure that iteratively you can improve your prompts over a period of time. So in this case I'm just going to go ahead and run these um and I can side by side kind of compare what is the output of my first version of my prompt versus my second version of my prompt.
And what this does is basically gives teams an ability to track what goes wrong as it happens in production, but then also have the tooling to then pull it back into a development workflow and iterate on that as as they go into as they go into production. So in this case, you can see here there's kind of a two different outputs here. This one, version one of my prompt took over 12 seconds, so pretty high latency. But it also, it looks like it doesn't actually call the right tool.
It doesn't actually pass in the right search column value versus my second version of my prompt actually passes in the right tool calling argument. So you can side by side kind of compare the two different prompts and see how how to actually improve it. Um, cool. What I'm going to move to next is actually talking about um so once you actually have these tool calling definitions and you're evaluating it, what teams then typically care about is how do I actually go in and evaluate the trajectory of the entire of the entire application.
So a and where this is really helpful are things like is there certain flows where my agents actually having a bottleneck? And a really good spot to actually do these types of evals is typically your planner agent or your orchestrator. I think there's different names for those, but in our case over here, what we see is that when users ask questions about, you know, typical Q&A type questions typically related to maybe eval, we're actually doing pretty good.
It's about 84% correct, only 16% incorrect. But when they're asking questions related to search, I'm going to zoom in if you can see this. Um, if you're asking questions related to search, that's typically where we end up being almost pretty equally off. We're about 55% correct, 45% incorrect. And so for us, this is actually the trajectory that we need to really worry about and think about how do we go improve that trajectory for our our application.
Um, and so this again coming back is like this is called a trajectory eval. You can think about it as across not just a single trace but multiple um you know multiple tool calls within the trace. Is it actually doing that in the right order in order to evaluate it? Um the last one that I'll kind of go through is is typically kind of a session eval. So you can go through and look at for example the back and forth responses between the agent and between the human.
And typically what teams will do is actually evaluate this at a session level to understand is the agent actually keeping things like the tone consistent or is it um remembering things from one session above to another session. This one obviously only has kind of one back and forth. So I'll kind of show you another example where we have um more of that back back and forth kind of use case. But when you do have these types of back and forths, you typically want to look and evaluate across the entire session.
Is it actually keeping track of um of kind of context across the conversations? So what we just went through so far is basically tool calling evaluations. We talked about trajectory evals and then we talked about kind of um uh session evals that you can do for your agent. Last one that I might just go through is kind of touching a little bit about the self-improving. So, how do you typically set up workflows so that you can evaluate your your agent?
For most teams as they're setting up these types of traces, you can actually evaluate the e the traces either using typically LLM as a judge or for some use cases you might want to do things like a code evaluator if it's something simple like um is the output of the agent fall between you know these three categories? Um does the output contain these keywords? But if there's anything more complex than that, we we've been seeing a lot of teams actually bias towards LM as a judge eval.

`20:11` Continuous self-improvement loop

In our case, what you can do, you can set up these evals with either a uh set of default evals that I have in the platform. So this one's kind of a default eval around tool calling or you can actually come in and define these eval. And what these eval do is basically run on top of all of your incoming traces so that as your application keeps growing data, you can then kind of continuously evaluate that application. Um little bit more of a again um production kind of workflow here.
So when I go back to kind of this self-improving, this is actually what we just did, which is we set up the traces, we collected the data, we set up a job to actually run those evals into production. When those eval production, what you can then do is automatically start to add any cases where it actually fails. So in this case, I can grab every single thing where the application's actually, let's say, hallucinated, and I can decide to go grab all of those, add those to a data set.
And what this does is it starts to create a data set of failures for me to then go iterate on. So I'm taking my data from production, moving that into workflows where I'm collecting data of data set of failures. Teams typically will then go in and annotate these data sets. So you end up creating almost like this human label on top of your data set. What would the right response have actually been? And then once you have that kind of annotated data set, you can actually use that to then go run experiments in our product.
Um and so in this case, uh I'll just go to the co-pilot kind of use case to show you guys what that looks like for our workflow. But we actually grabbed every single use case where um the agent wasn't kind of performing well. And you can actually see these are some of the examples that we've collected. And what we've done is we've actually run different experiments over time to iterate on the tool calling definition or iterate iterate on the actual argument correctness.
And you do these iterations, you track these experiments, you evaluate your experiments. Once you have a really good prompt that you've experimented on, that's typically what gets deployed into production and gets pushed into the real world. Um, and teams will store all sorts of things like the prompt that they iterated on. You can track all the different versions of the prompts in in the product. Um, and so you end up having this whole workflow where you go all the way from development, you know, into production and then identify those use cases where it fails and then go back into development mode with that data.

`23:17` Prompt engineering and iteration

Um, any questions so far? Does this make sense to folks? Is this uh a kind of workflow that you guys are doing today? Cool. Cool. Um, awesome. Well, maybe what I can spend some time doing right now is actually talking through a little bit of how teams do these type of iterations in the product. Um, you can do this using CI/CD or for a lot of nontechnical teams, what they might just do is literally just upload a data set into the product.
So, you can set up a workflow where you upload a CSV into the into the product. In this case, we'll just make this a super simple example. Um, I'm actually running an experiment here if you, you know, just to remove all sorts of context out of this. But in this case, just just for folks to see, I just have a data set of simple movie descriptions and I have simple what I wanted to actually guess is the movie title here.
Um, what I'm going to do is go ahead and show you guys how you do these types of prompt iterations iteratively in the product. So, I'm going to go ahead and pull up this data set. I just wanted to I could set a simple thing like guess the movie in the original description. Uh, this is actually my my data set. And what I wanted to do is actually guess the movie that's actually in in the data set. So going back to my data set, this is actually what it looked like.
These are the descriptions. So I wanted to actually guess these prompts over here. So I'm going to go ahead pull this in. We'll write the prompt again. Guess the movie in the original description. I'm going to go ahead and then just run uh oop guess the movie in the description. I'm going to go ahead and actually just run this um and see what it actually gives me over here. You can see here I ran my actual prompt. Uh and then it actually just went ahead and saved this off as an experiment in in the product.
So, let me pull up kind of what this looks like over here. This is my actual experiment. This is actually the output of my experiment. In this case over here, it's actually generating descriptions. it's not actually generating the actual title of the movie. So, what I might do is actually go back and iterate on my prompt here. So, I can go back, go back into my prompt playground, go back and pull up, let's say, my descriptions here, and then try a different prompt.
So, guess the movie in the description. Um, let me pull back up the prompt here. um with just one word. So these types of prompt iterations that teams are actually doing they end up doing you know you could do compare side by side all of these iterate them in the product and then run evals on top of these prompt iterations so that you can actually evaluate the outcome of each of those prompts. And the key thing here is that you don't just want to do this on your application prompt.
You also want to do this on your actual eval prompt in in the application. Um, so, uh, just wanted to give you guys a little bit of a workflow of how you can think about iterating on on your use case, iterating on your prompts, um, and test this out with kind of both Arise and Mosaic ML. Uh, I think I have about eight minutes or so left here. So, what I might do is just open it up for questions and see if you guys have any questions about emails.
Yeah.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.