AI Governance Best Practices: Extending Data Controls to Agents

Summary

Tim Loritz from Databricks and Cheyenne Mahanti from Thoughtworks explain why traditional governance models break down for AI agents, whose tools, execution order, and side effects are non-deterministic and resolved at runtime rather than predetermined.
This video covers five essential governance primitives required for agentic systems: identity for attributing actions to responsible parties, capabilities for short-lived scoped permissions, sandboxing for execution containment, observability for end-to-end tracing, and resource accounting for real-time cost control.
Databricks Unity AI Gateway and Agent Works provide platform-level governance that builds on existing data governance foundations, enabling organizations to move fast with AI while maintaining security, compliance, and cost control.

AI Governance Best Practices: Extending Data Controls to Agents

As AI evolves from single models to multi-agent systems, governance models must evolve too. Traditional approaches break down when agents dynamically access data, tools, APIs and multiple models across platforms. Building governed agentic systems requires a foundation: extend data governance principles from the data layer to the runtime layer, with unified controls across data, models, and agents.
Learn five essential governance primitives for agents: identity (who is responsible), capabilities (short-lived, scoped permissions), sandboxing (execution containment), observability (end-to-end tracing), and resource accounting (real-time cost control). Discover how Thoughtworks and Databricks are building AI governance into enterprise platforms, including Unity AI Gateway and Agent Works, enabling organizations to move fast with AI while maintaining security, compliance, and cost control.
🤝

Chapters

FAQs

Why doesn't traditional governance work for AI agents?

Traditional governance assumes known tools, predetermined execution order, and deterministic side effects, but AI agents operate differently — tools are not fully known ahead of time, execution order is resolved at runtime, and every interaction is probabilistic by nature. Governance systems designed for classical software cannot be directly applied to agents without modification.

What are the five AI agent governance primitives?

The five primitives are: identity (attributing actions to responsible parties), capabilities (short-lived scoped permissions), sandboxing (containing agent execution), observability (end-to-end tracing and auditing), and resource accounting (real-time cost control). These primitives are designed to bring the same discipline to agent governance that operating systems brought to multi-program computing.

How does data governance serve as a foundation for AI governance?

The data governance layer — including Unity Catalog access policies on tables, models, and tools — provides the substrate on top of which AI agent governance can be built. Organizations that have invested in strong data governance are better positioned to extend those controls to agents at the runtime layer.

What tools does Databricks offer for AI agent governance?

Databricks provides Unity AI Gateway for governing LLM usage and costs at the platform level, and Agent Works for enterprise runtime governance. Both are designed to help organizations move fast with AI while maintaining the security, compliance, and cost control needed for production deployments.

Full transcript

[00:07] Everybody, I'm Tim Loritz. I'm at Databricks as a lead AI product specialist. I'm joined by Cheyenne Mahanti. Hi, I'm Cheyenne. I'm the chief data and AI officer at Thoughtworks. And uh we're excited to co-present on a topic both companies are passionate about and we've been working on, which is AI governance.
[00:24] So, um I suspect that almost everyone in this room uh is on some version of the same path. You've got an agent probably working in a demo environment. Maybe you've, you know, done kind of like the all hated POC. Uh and now someone is asking the real question, can we actually put this agent
[00:41] next to real customer data or managing real money or doing something real? Um and that's that's usually where people go a little quiet. Um it's it's not that the technology itself doesn't work. Uh it it obviously does, but it's all of our mental models, all of
[00:57] our systems that we've historically put in place to manage software uh kind of break under a new set of assumptions. Um the tools that agents use, for instance, are not necessarily known well ahead of time. The exact execution order
[01:14] is not necessarily known well ahead of time. It's resolved at runtime. Uh every single interaction is uh probabilistic by nature, non-deterministic. So, the side effect is that uh we can't really govern agents in exactly
[01:32] the same way that we've governed all other software, but I think the good news is that we've built largely the same primitives for the classical software world. We're coming from a world where many programs
[01:47] first shared one machine and none of them could be trusted. So, you know, at that moment when that happened, we as people didn't just write them a nice little memo and be like, "Please behave." Uh we designed systems. We designed primitives that were baked into
[02:03] the operating system. Things like identity, things like runtime governance, things like you absolutely can access certain like this part of the file system or you absolutely cannot. Um agents sharing an enterprise are like part of the same problem, but
[02:19] it's up a level, right? Uh so the good news is that the foundation already exists. And it turns out that all a lot of this thinking has existed, you know, previously in the data estate.
[02:37] Yeah, so here we are at Data and AI Summit, right? Um so we are starting with data. We're going to build up from there. Um a lot of you I assume are here though because you want to talk about AI. That is the hot topic these days. Um what we hear a lot from our customers is is that they've invested very heavily
[02:52] in governing their data layer. Right? We've been doing that for years. That's been a topic of discussion at this conference and others for a long time. And and a lot of customers are just now at the point where they feel like they have a good handle on that. And then along comes this gen AI, the ChatGPT moment. What do
[03:08] you do with that? Well, the good news is what we found is that with the foundation of governance that we're doing on data, so with things like uh tables, files, functions, UDFs, uh even machine learning models that are a little more uh contained than LLMs, uh
[03:25] those you know, what we've done for those generally extends to AI, which brings a whole new set of what we call securables, right? So LLMs are inherently different from classical uh machine learning models, right? MCPs are new way of exposing tools uh that's a
[03:40] little more flexible and perhaps open to uh security issues than uh than just functions. Skills, like even a year ago, who knew about skills, right? That's the thing we have to think about now. How do we govern those? And then agents themselves, can you make an agent an actual securable? Um so, that's
[03:58] the objective is we want to make sure that those securables can follow the same sets of great things that have been you've been doing already in your data estate. We might need to add a few more controls. We'll talk about that in just a minute. But, the good news is if you've already governed your data estate, then you're in a good position to capitalize on AI governance as well.
[04:15] Yeah, so let's let's talk a little bit about the runtime and the substrate. Um I I I think it's important to kind of like call out the fact that like half of this already works. If you think about data governance, right? This has been a topic again that all of us have been thinking
[04:31] about in some way or another. Uh we have sophisticated tooling, we have ways of working, we have the concept of data products, like etc. etc. Half of it already works, but the data estate isn't where an agent actually runs. The agent runs in its own
[04:48] execution boundary. It's composing its next steps as it goes. So, that's the half that's new. And and usually that's the half that um you know, nobody's really properly governing just yet. So, so the real picture is is two halves that have to
[05:05] hold at the same time. The runtime, where the agent actually executes, and the substrate, the data and tools that it reaches. If you govern one and you just leave the other, then the gap is wherever you stopped
[05:21] looking, right? Uh but underneath both are the same five primitives as what we kind of independently discovered. Uh there's identity, there's capabilities, there's sandboxing, execution tracing or more broadly observability,
[05:37] and resource accounting. So, let's let's talk through each one. Um so, the first is identity. And And this sounds like a very innocuous one, but I I think it's actually one of the most important ones.
[05:53] What constitutes agentic identity? If you think about it, like uh a lot of the time there there's sort of a quick answer, like oh, agents will just act on behalf of a human all the time. It It inherits the human's identity.
[06:09] But that's not always the case, you know, think about the cases where you might actually run a long-running agent, maybe it's doing ticket remediation or something else, where it's not actually tied to any one person's identity. It has an identity of its own. Or perhaps you're talking about um kind
[06:26] of a a multi-agent or a sub-agent cascade. Right? At what point is a new identity introduced? Or is it the same identity that cascades through that whole chain? Right? This is actually at the core of a lot of the architectural decisions you
[06:41] start having to make because this cascades in a lot of different ways. You start talking about, okay, can I can I keep keep reasoning about the same IAMs? Uh do I have to think about something different? Do I Do I use service accounts or are service accounts not enough? Like now you have to start
[06:57] really thinking about how that plays with the rest of your architecture. Uh so, what we're trying to avoid in like actually addressing this head-on is you don't want to have a problem where an agent goes off and does something and you can't trace down exactly how to
[07:13] remediate it, you know, who who can you hold responsible on the one hand, but also who can help fix it? Um these are two very important things that any functioning company are going to need. Uh and so we think identity is kind of the first place to start. Um so,
[07:29] ultimately, if there's there's no person, there's no run, there's no log, if there there's just a service account entry, um there's no one really to ask. And so, identity is what guarantees that there's always someone there uh to help.
[07:49] Yeah, we like to think about identity as what you need to know for the agent in order to take action. Um and it comes down to classic principles of risk management, right? What's the blast radius of what the agent can do? Uh I assume most people here probably use the chat GPT app, right? Like that is just limited context
[08:04] unless you connect it to something crazy for the from your your your personal world. Um Anybody here actually install Open Claw? Just curious. All right, a few brave souls. That opens up a a whole other can of worms because now it can access more and more of your stuff, right?
[08:21] Um so the blast radius is sensitive, so it's high on the left on the on the vertical axis, but not so much on the right because it's just you. But what happens when you have an agent that can actually access sensitive resources inside the enterprise? Now you have a
[08:37] whole new set of concerns to worry about. This is obvious, yes and no. But I Cheyenne mentioned, you know, agents sometimes do need their own identity. Totally agree with that. But one of the things that's been top of mind for us as we've rolled out Excuse me. Um agent substrate support
[08:53] is that this idea of on behalf of needs to be a primitive. It needs You need to be able to trace the end user, right? Maybe even some tags about what their use case is all the way through that chain so you can see all the things that they do and take action with the the permission set of that user. Maybe more
[09:10] or sorry, maybe even less than what that user has. So the the agent should never have more scoped permissions than what the end user themselves had. So this brings us to pillar two, which is capabilities. And that's
[09:26] you know, basically to say, okay, so so once once you can name who is acting, the the next question almost asks itself. What What they allowed to do fundamentally? And and the instinct is is obviously to reach for what we already know, what we already have,
[09:42] which is often roles, right? You think about our back style implementations. But the way that I think, you know, we think about this at least, is that agents don't really have permanent roles
[09:58] quite in the same way that humans do. You know, obviously our roles change as individuals. We will will slowly drift that dimension, if you will. But take a deep research agent as an example. You point it outside of your four walls,
[10:13] you're probably fine with it accessing an open like some some some outside models, some some vendor models, some open MCPs, like it's all fine. But then someone might look at that output, that that research paper or whatever like output that came out of it, and say like, I like the way this
[10:29] thing did its reasoning, I like the way it did its research. I want to take that capability, and I want to point it at our finance stocks. Right? Now all of a sudden the same agent is being used in a wildly different context that then has a wildly different compliance uh posture, which
[10:46] then implies a wildly different capability surface. Which now means that you can't really think about it as an agent holding a role and therefore having permissions. You really do have to think about it as short-lived permissions grants that are
[11:02] designed for one thing and one thing only, and and it's aligned to a use case. So basically what we're saying is that fixed grants cannot work, whether they're attached to roles, whether they're fixed in and of themselves. Uh really you have to think about these
[11:18] capabilities that are granted for specific tool, specific data sets, really wrapped up as uh a specific use case. So the whole idea is lease the capability for the purpose, and there's nothing for an injection or some other
[11:34] like attack vector to reach on to. Ideally, you scope to exactly what is necessary in that particular moment. And again, this becomes an architectural consideration. How do you design a system such that you have that ability to scope down before you ever run
[11:49] anything? And maybe to put slightly finer point on this. Uh sometime this is principle still apply. Like least privilege, good. Right? We want to use that here in the AI space as well.
[12:04] Um traditionally this meant dealing with, you know, you know, read, write, execute on files, right? In the file system. Which is good, but that's not good enough for AI. We need to extend this just a little bit. Um for example, if you are exposing an MCP server
[12:20] for GitHub and you want to provide that to your to your developers so they can automate workflows. Um you might want to limit what the intern can do. Right? And now, some of this you can do directly in GitHub, but some of it you might want to
[12:36] do at a finer grain for this project just on the MCP server level. Um so you don't want them, for example, merging a pull request. Um you might want to make sure there's human in the loop on that. So some things that we extend here are what is the identity of the user,
[12:53] which tool are they accessing, what's the context, right? There might be additional context provided that's use case specific where they can I can't take a certain action. Lastly, too, is what action can the agent take? So like going back to read, write, execute, that's good, but there
[13:09] needs to be some others, right? So uh the way that we've built this out is to think about having allow, deny, and also ask, right? So you inject a human in the loop where it's critical to have that oversight. Yeah, so I think related to this, right? So we're
[13:25] talking about capability scoping, we're talking about like what you can and can't do. Um there's a super important thing here, which is becoming more important by the day, which is sandboxing. Um I I think probably everyone in this room has by now realized that um just pure LangGraph
[13:43] style, almost like state machine flavored agents are not really the thing that's going to push business, not just productivity, but like transformation to the next level. Uh you start getting into, for instance, like uh the topic of the day is coding agents, right? Super
[14:00] powerful things that can do arbitrarily complex things, like in order to have a proper coding agent, you need you assume file system access. You assume the availability of certain interpreters, certain coding tools, and so on. Um but that's not just for coding, right? Think
[14:17] about uh as Tim mentioned earlier, claw-like agents, right? Imagine that you could spin up um you know, a claw for each of your employees, right? Each of your coworkers. But do it in such a way where uh the blast radius is well contained.
[14:32] You know, they're only able to see the things that they're meant to see at that particular moment. And they have no visibility, no access to anything else. Right? In order to do that, it is fundamentally an architectural question. There's, you know, the actual
[14:49] act of scoping the capabilities, you know, through permissions, through tags, etc. But there's also the physical reality of like we're going to run this thing somewhere. It needs an environment within which it's running, and you have to have certain guarantees up on that environment. So, uh
[15:05] a very, you know, concrete example here is um we've run into this in the past. Let's say you have an agent that is able to do a whole bunch of things. It's able to touch sensitive data, you know, a database that has PII. But it's also able to touch non-sensitive data, you
[15:21] know, a database that doesn't have PII. Let's say it goes and it's doing something and it reads from that sensitive database. Do you want to be the arbiter of what precisely it read and therefore what precisely it can write? Like that's
[15:37] that's a recipe for disaster. Instead, it's easier to just say, "Look, I know that you you read something from this sensitive location. Therefore, I'm going to taint you. You can't write now to a non-sensitive downstream location, right? So, it's these realities where you want to design
[15:54] this at a system level. You want to basically eliminate entire classes of problems from ever happening. Uh that way you can actually deploy agents, build them without having to, you know, kind of wake up in a cold sweat uh thinking like not not what you did, but
[16:10] perhaps what some of your coworkers might have done who may not be as uh familiar with agentic development as you are. Um so, yeah. Yeah, what Ryan said about uh tainting is really powerful. And we're excited to see how that gets used uh by our customers. I think that's a uh uh
[16:26] the right way to do things. We can also kind of do a little throwback here, right? So, this conference used to be called Spark Summit. Anybody been to Spark Summit years ago, right? So, it's evolved. It's Data and AI Summit now, but Spark was a you know, it still is a thing and we can borrow a lot of the same lessons learned
[16:42] there for sensitive data processing now in the agent space. Um so, for example, like when Spark scales out, it's going to spin up unprivileged containers. So, the data, right, there's no root access. Uh things are ephemeral. Uh nothing's persistent to disk. Processes drops. As you do
[16:58] that, too, uh it turns out that compute's expensive. You don't want to waste your agent cycles, especially because of the LLM calls, which are really expensive. So, you can make sure that the compute is durable. Made for some announcements about that this morning. Um at the next level, and pretty much
[17:13] all of our enterprise customers do this, is that there are very tight network controls around um calling out and and pulling data back in to from outside your outside your network, right? So, super important to get that right. Everybody has network access control set up.
[17:31] The last layer would be what what I just mentioned a couple of slides ago where okay, let's say everything passes your what your agent's asking for is within the approved network scope for your enterprise and it goes out and accesses an external MCP server. Well, which tools can it use and in what
[17:48] context? Right, so it goes back to the tainting idea. So, we want to make sure that all those rule sets are applied to the boundary for for the this tool gateway. So, for example, um you your company might be using like a private PyPI mirror, right? And there's
[18:03] a lot of companies do this. So, if you're going to go um grab a package, it needs to come from there, not the public one, right? And this has come to a head with some libraries this year. You or if you just try to reach out to some random website and download a tarball and install that locally, like
[18:20] okay, network controls are going to check that and fail it. Uh on the other hand, like at the tool gateway layer, couple examples, like for us internally, we use you know, all kinds of automation with agents to get stuff done inside Databricks. So, for example, like automating a whole
[18:36] bunch of messaging on Slack, that's a good thing, right? And people should be able to do that within bounds. So, that's going to pass the check. But, if now you're connecting to your ERP system trying to place an order for something, okay, that should not be something that any user can do. So, if that MCP server's there, you better have some
[18:51] tight scoping on what users can do with it. So, sandboxing has a few different layers to it and definitely something I think we'll see a lot of movement on this year. And I think this is probably going to be the least controversial if any of the stuff we've already talked about is at all controversial, but this is probably the
[19:07] most benign. I think everyone here probably agrees that observability is important. You know, what what cannot be observed cannot be fixed, cannot be tested, cannot be really used. Um so, ideally
[19:23] observability is not a bolt-on. It is it is threaded through every single aspect of the system, whether it is an actual agent taking uh action on something or even the platform of the system above it making a judgment that an agent cannot do something, right? So,
[19:40] from the agent perspective, it's a no-op, but from the system perspective, you actually do need to see um what happened and why and how and why something was denied. Um so, basically the punchline here is uh we firmly believe that uh a single pane
[19:56] of glass, if you will, kind of like a a a a single way to see not only the agent view of the world, but also the system view of the world is pretty critical and and getting kind of like that end to end execution tracing is super important, not just from, you know, action to
[20:13] action, agent to action, but end to end, how an entire run kind of started and ended. Yeah, and no controversy here, either. I agree with that. Um tracing's not a new idea, right? You've been doing this in
[20:29] software development forever, uh but it adds extra value in the world of agents. And so, we place a high priority on making sure that tracing is easy to set up and easy to process on the back end. Um so, in the sense of tracing, if you haven't done this with agents before, just a really quick 101,
[20:46] what you're seeing on the left is a essentially a trace view on the left and the the chat uh with the agent on the right. So, tracing's going to capture every LLM call, every MCP call. It's going to show you the messages that were passed, the context, the tokens that were consumed, and um the time
[21:02] taken. So, incredibly information-rich. And so, why do you want that? Well, it's about trust. So, you need to be able to deeply inspect the agent's behavior so that you can trust it. Uh a good example would be hallucination, right? So, you might get a weird answer at the end or
[21:18] an answer even that makes sense, but you also want to see the citations, the sources, and not all systems will surface those to you. The tracing can tell you exactly which systems which resources were queried to grab context, and then whether that context was fed to subsequent analytic
[21:35] calls to do reasoning. Um if you don't see it, then that's a problem. You need to fix your agentic deployment. So, that's for for trust of the agent itself, right? For the developer lens. On the right, we're talking about sort of the security or the data owner, data
[21:50] steward lens, right? You want to make sure that uh the the permissions you've given to access your resources, right? Your uh uh your your gold inside your enterprise, that those are being enforced. This goes back to identity as
[22:06] well. When you see the audit on the access of those resources, it should show a couple things. It should show the agent, but also the the or- originating user's identity as well, which is what you see in uh those logs. Um ideally also, you have some notion of lineage. So, if someone uses an agent to create a
[22:23] new object, and then uh ship that around, that needs to have some notion of where it came from and what resources were consumed in that. So, we're trying to stitch uh again, these things we've used for data in the past, now to agentic workflows, not just uh uh data
[22:38] processing workflows. Yeah, so this is um pillar five, and I think this is probably kind of a a thing that everyone in this room has gotten sweaty about at some point, uh token costs, right? And how do you manage them? As more and more
[22:55] of these organizations are becoming AI first, essentially you're saying that your organization eats tokens for breakfast. So, how do you manage that? It is is something that's like top of mind for everyone. So, um here's the thing. We're We're We're so
[23:11] used to spend being something finance reconciles at month end that it's easy to miss. For For agents, cost is a control surface. It's It's spent in real time. It must be managed in real time.
[23:26] Um an agent doesn't spend like a query does, right? It can loop. It can retry a plan. And And those costs may not just be in tokens. Like some MCP servers that you integrate with may have a cost gate against them. So, as things are retried,
[23:43] as they're replanned, you you not only compound the cost of the tokens, but also potentially the cost of reinvoking so some of those uh some of those tools. So, if all of that is true, then
[23:59] budget cannot just live in a dashboard. Budget must actually be a control surface. And it's really easy to say like, "Oh, for this agent, um I want no more than so many tokens to be spent every month." I think that's like at
[24:15] this point most people have wrapped their heads around that. But in a world in an enterprise where you have some number of long-running agents, you have some number of uh coding sandboxes that people have spun up, you have some number of claw agents that are just
[24:31] responding to things, uh some number of sub agents that get spawned out of all of those things. Like, how do you manage the underlying cost? And what is the What is the unit? Is it Is it dollars? Is it tokens? And And how do you also
[24:47] think about it in a world where you have in a lot of cases model fungibility. You might be all in on one particular model vendor. Or if you're like a lot of other companies, like you have a portfolio of such model vendors. So, across all such models, across all such tools, across
[25:04] all such things, how do you make sure that you're metering effectively? Right? In order to be able to do that, then it implies a certain again, architecture is probably the the name of the game here, right? How do you actually make sure that you can see all of that? That implies that you have a
[25:21] system that is designed to see all of it. Um so, kind of the idea here is that you should be able to limit by a whole number of different dimensions. You know, some of it could be absolute cost, some of it could be burst cost, like you're not
[25:37] allowed to burst past a certain amount, you know, per run. Some of it could be concurrency limits, like I don't want more than this many agents running on this particular node at one particular time, right? Or it could be delegation
[25:52] depth, you know, how many agents am I allowing to kind of like cross-delegate, so I don't end up in this like infinite, you know, pile of agents at some point, you know, turtles all the way down. So, like these are problems that we have faced
[26:07] a lot. You know, as we go in and we do these AI transformations with companies, we've run into this problem many times, and so we've we've spent blood, sweat, and tears trying to like undo these issues, and we're like, "Never again." Uh so, we we just kind of forth like
[26:24] we're very forthright with we believe that in order to solve a lot of these problems, it is not just a bolt-on governance activity, it is very much you have to design your system from day zero to support all of these things at the same time.
[26:45] And maybe just to give some concrete example, we also at Databricks have had to solve this problem ourselves, right? And ideally, you solve it using our own platform, which is what we try to do. So, it's shaped the way that we roll out AI to our customers. Um in the early days, like 2023, a lot of organizations
[27:03] used a single model provider. Remember that? And we used to do that. Uh not so much. We still I have to talk to some customers these days who still just use one. Uh I wouldn't advise doing that, like just because of the way the industry's heading, it's better to have, you know, uh optionality.
[27:18] Um if you're using just one, what's nice is that most of those providers have built-in cost observability and control, right? So, if you're just riding that one train, it's pretty easy to do. Um the challenge becomes when you're starting to span providers. Uh
[27:34] and again, most organizations are. So, how do you retain observability and control? Observability may not be too hard of a problem to solve. You can just, you know, dump your metrics out from each platform and put it in some database somewhere and put a dashboard up and say, "You Hey, I can see what I'm
[27:50] spending across all my providers." Great. The problem is the genie's out of the bottle, right? You haven't solved the control. So, really what you need is a layer, the substrate, where you have both observability and control across providers, that doesn't slow people down, right? Because your developers are
[28:06] going to be asking for more and more models and the best models. Obviously, you want to throttle what they can use without hampering their productivity. So, that should all be built into the way we think about rolling out uh control. Um Ideally, too, uh it needs to be at a
[28:23] level that makes sense, right? So, you might assign a budget for a use case, right? And you want to attach that to outcomes. You need ROI measures. So, that notion of a use case, right? That tagging needs to flow through the whole system, so that at the end of the day, if it whether it's cost savings or
[28:38] revenue generation, you can take your uh output metrics compared to what it cost with AI to do it. Um without that, it's going to be hard to justify uh further investment in uh your your AI initiatives. Yeah, this this one is um
[28:54] one that again we we ran into in practice. So let's let's imagine for a second that you implemented everything that we just talked about, right? All of those pillars. Let's say you did all of that. And you have really clever gates and guards against certain agents from using
[29:11] certain tools and so on. And then someone, you know, unknowingly created an agent that will eventually violate one of those guards. Right? But it's it's buried in a whole mountain of like Python code or
[29:26] TypeScript or something. Like it's somewhere in there. It's going to eventually potentially access a tool that it probably shouldn't. Right? Let's say it's like a really sophisticated agent. It's going to run for a long time. It'll run for a couple hours. It'll touch a bunch of downstream systems. It'll go mutate some databases
[29:43] here. It'll it'll remove some records. It might like trigger something downstream that will cause a time series entry to be entered. Like all of a sudden you do all of that, right? And then halfway through execution or maybe like a couple hours in execution, it hits a wall.
[29:59] It's like tool unauthorized. Tool unauthorized. Like now if it can't finish its flow, then it's possible that all of the changes it made along the way no longer hold the same context. Now you might have to be in the business of tracking down all of the
[30:15] things that it decided to do. Right? These are not static flows, right? So at every instance, it was able to make a choice and it mutated some data here. It it, you know, read some data from here. It did some other stuff. So now you have to go back and basically rewrite
[30:32] history. And that may be easy enough in certain data environments, but we all know that we live in kind of like a heterogeneous world. You've got some things that are easy to revert and some things that are less easy to revert. So, our opinion is that along with this, and
[30:48] this is super important, you have the ability to understand is it even possible for my agent to do the job it's supposed to do? Is it even possible for it to run end to end? And if it is, then you should be allowed to run it.
[31:03] But if it's not, then ideally you know that ahead of time. And you're also told how to remediate that before you ever execute anything. Um if you can't tell, we ran into this. And it was our our team that had to go and figure out how to hunt everything down. And once again, we were like,
[31:19] "Never again." Uh so, now this is becoming an architectural primitive for us. And so, I I want to also kind of like land on a on a quick case study here. Um so, I neglect I I introduced myself, I
[31:35] neglected to introduce the company. Um so, I work at a company called ThoughtWorks. We are a professional services organization. Um if you know the name ThoughtWorks, it's because you know we made Agile popular. We also invented CI/CD. We coined the term microservices. We
[31:50] invented the concept of data products. We We've had a long and storied past in the engineering world. And uh these days we're very focused on how to bring that discipline, that engineering best practice mentality, to the data and AI
[32:05] world. And as part of this, we we work with a lot of our clients uh to help get them many steps closer to becoming, you know, agentic first. How How do we rewire their organizations to work the right way? So, um OAG is a really interesting, at least first step of that
[32:23] story. Um you know, they they are the data backbone of global aviation. So, that means real-time status, schedules, and pricing from more than 700 different providers, all feeding airlines, airports, and travel platforms worldwide.
[32:39] The status domain within OAG ran on a legacy on-prem monolith with data quality handled the old way, manually in silos, and after after something had already gone wrong. We rebuilt all of that on Azure
[32:56] Databricks. They didn't write a better data governance policy. They built governance into the platform. And as a side effect, anomalies are caught before a customer ever sees them. Uh new sources can get onboarded in
[33:12] hours through automation, not weeks of hand rolling. Uh quality issues that took five manual steps to resolve now takes two. All of this is basically the first big step. It's It's getting the substrate right. It's
[33:28] how do we get all the tools and the data to be ready for the eventual agentic future. So, this is kind of like the first step that most everyone takes. It's It's how do I get the substrate in the right place with the right things and the
[33:43] right governance apparatus so that we can then latch onto that from the runtime perspective. Okay. Not surprisingly, I'm going to show you a Unity AI Gateway slide. You probably saw this already in the keynotes yesterday and today. There are
[34:00] breakout sessions that go deeper on this. This is not a product pitch, okay? I'm just pulling this up as sort of a capstone to everything that we built up to in terms of the primitives that they need to be implemented in the platform. Most of those things are in, some are still in flight, but we're very excited about the future because we feel like
[34:16] we've aligned to first principles on this. One more thing I'll call out, too, is whether you're using Unity AI Gateway or some other product, it's important to have interoperability. And so, we've kind of built that in as well. So, for telemetry and logging like using OpenTelemetry,
[34:32] uh for exposing your agents or sorry, exposing tools and resources using MCP, right? And keeping up with the spec. Authentication, everybody loves it. Uh so fun to do. But making sure we're using OAuth standards throughout, so it's easy to uh to inspect and authenticate our identity.
[34:48] Um so again, we built up to this. This is the capstone and uh we'll let uh Cheyenne talk about his version. Yeah, I mean, look, we also built something unsurprisingly. Um we call it Agent Works and really the way to think
[35:03] about it is um we live in the runtime layer, right? We live at the agent execution environment. Um we want those agents to connect into the underlying substrate. Um and this
[35:19] all kind of happened uh through happenstance. I think Tim will will talk a little bit more about how all of this kind of came about, but we discovered that our friends at Databricks have been thinking about governance in almost exactly the same way we have. Uh they clearly lived the pain, lived through
[35:36] kind of like the remediation, and came to roughly the same set of conclusions, but from a different perspective. They grew it out of the data substrate, and we grew it down from kind of like the agent execution world. Um so everything that we talked about, all the pillars are encapsulated in both sets of of
[35:53] products, whether you're talking about Unity AI Gateway or you're talking about Agent Works. We cover a lot of the same things, but it's it's from two different sides, and so there's this really interesting better together story. Yeah, so you can see the QR code here, which you can scan for an ebook where
[36:08] we've uh um made these uh principles maybe a little easier to read. Um but yeah, it was a really serendipitous encounter. I mean, we do work with ThoughtWorks as a partner regularly, but um we were surprised to find just how eerily similar the stories were in terms
[36:24] of both the customer engagements and how we were planning to implement the needs, right, the requirements in the product. We've both been building fast and furious this year and really happy with how things have turned out. The whole point here again is to make sure that
[36:40] it's like the old mantra of you know, slow down to go fast, right? When you begin with this governance mindset, then you can scale when you deploy AI across your enterprise. Skip it and you're going to get blocked whether it's on security or cost or something else. But if you start with these principles,
[36:57] and use the product the right way, then you can go fast. Yeah, I think the the big punchline here is substrate and the runtime together make the entire enterprise environment, right? Only if you only have one, then you only have half the story. So, the
[37:13] whole idea here is that between us and Databricks, you cover the entire spectrum. And if you have a fully governed environment, it creates safety, it creates trust, it allows folks within the organization to just lob as many agents as they can at the
[37:28] system and you trust the system to say what can and cannot happen. So, we're hoping that through this you recognize that governance is not just an enabler, it is truly an accelerator. It will allow everyone to go faster. It used to be a dirty word, but now
[37:45] hopefully it's a little bit more fun. So, thank you so much. Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.