Getting started with AI Governance on Databricks

Summary

Agent sprawl—thousands to tens of thousands of agents touching sensitive systems across every function—makes traditional governance approaches inadequate, and the Databricks Data and AI platform addresses this through contextual access control that unifies AI governance with data governance in Unity Catalog.
On-behalf-of context attributes distinguish user identity from agent identity so access policies can restrict what an agent can do even when the user it acts for would have broader access, and service policies control which specific tools within an MCP server an agent is permitted to invoke.
Unity AI Gateway provides intelligent routing to cost-efficient models, tokenomics capabilities to cap AI spend per user, team, and model, and unified observability through a single Delta table of AI traces across all platform assets.

Getting started with AI Governance on Databricks

As AI agents move from prototypes to production, a new architectural layer is emerging: the tool gateway. Models alone aren't enough — agents need secure, real-time access to data, APIs, and enterprise systems. Integrating these tools consistently and safely has become one of the biggest bottlenecks to adoption.
In this video, we explore how Model Context Protocol (MCP) and tool gateways redefine how agents interact with the world. MCP standardizes how agents discover and use tools dynamically, while AI Gateway acts as the enterprise control plane to govern access, manage credentials, and monitor usage across every tool and model. Together, they replace fragmented APIs with a unified, governed interface — letting agents securely connect to managed, external, and custom MCP servers and select tools at runtime.
Through real architectures, you'll see how to design agent systems that are flexible, interoperable, and production-ready across internal data and third-party systems.
Talk By: David Nasi, Director, Product Management, Databricks ; Stefania Leone, Senior Director of Product Management, Databricks ;

Chapters

FAQs

What is agent sprawl and why is it a governance problem?

Agent sprawl describes the proliferation of thousands of AI agents across an enterprise, each touching different data systems, APIs, and tools—often without centralized visibility or control. Traditional data governance was designed for a small number of known users accessing defined sources, and it breaks down when agents can join data across many systems simultaneously and take real-world actions.

How does the on-behalf-of context attribute work in Databricks AI governance?

The on-behalf-of context attribute allows the Databricks Data and AI platform to distinguish between a human user's identity and the agent acting on their behalf within the same request. This enables policies that limit what an agent can do—even if the user it represents has broader permissions—preventing agents from taking actions that a human with the same access would know to avoid.

What are service policies in Unity AI Gateway?

Service policies in Unity AI Gateway restrict which specific tools within an MCP server an agent is authorized to invoke, providing fine-grained control beyond simple allow or deny at the server level. This means an organization can expose an MCP server to agents while ensuring they can only call the subset of tools appropriate for their specific task or role.

How does the Databricks Data and AI platform track AI costs and model usage?

The platform's tokenomics capabilities allow administrators to set and cap AI spend per individual user, per team, and per model family. Intelligent routing automatically directs workloads to cost-efficient models when the task does not require a frontier model, and all AI interactions are logged to a unified Delta table for observability and auditing.

Full transcript

[00:08] All right. Welcome everyone. Uh, I just want to thank you for joining us at the last day of the conference in the afternoon to talk about governance. Uh,
[00:20] I'm David Nassi. I'm a director of product on our AI team and today I'm also joined by Stefania Leona, my esteemed colleague, uh, who's a senior director on our governance team. And so today we'll be talking about
[00:32] getting started with the AI governance on Databricks. So, what will we cover today? In order for us to frame the problem a little bit, we'll we'll talk about where
[00:45] does AI governance uh, fall short, uh, today. Then we'll talk about what is good AI governance can look like and what are we at Databricks doing to help you get
[00:58] there. And then we will also talk about, um, actually we will show you how it all works in practice and Stefania will show us a demo, um, that will kind of get uh,
[01:10] put everything in place. A few years ago, enterprise AI were was essentially just, you know, a central ML
[01:24] team, uh, you know, building models and so forth. Uh, but that's not the case anymore. Every organization we talk to are embracing AI, obviously, and, uh, are
[01:36] gaining a lot of productivity gains with that. They're using AI for coding and engineering tasks, for uh, marketing, for, you know, knowledge work. Uh, essentially AI now permeates every
[01:49] organization, uh, and every function. Uh, and that's great. Like there are so many benefits. But what that causes is what we call agent sprawl. So, essentially now you
[02:02] have agents thousands, if not tens of thousands of them, living in the organization. Um they touch all sorts of systems, data,
[02:14] um and and now it becomes really hard to um actually govern that. And traditional governance approaches have are inadequate to actually have a governance
[02:27] strategy over over this sprawl. So, let's talk a little bit about the what we define are the perils of agent sprawl.
[02:42] So, the first peril is control. Um and control actually is how do you know and how do you um deal with these agents accessing all sorts of sensitive systems? So, you can see here how,
[02:54] you know, with no malicious intent, um recently there was a case of an agent going trying to solve some problem and deleting a production database. Um agents are focused on solving, you
[03:08] know, problems that you give them to solve, uh but then they don't always have the judgment of how to do it. And so, they can do it against company policy um and and against um kind of the the judgment that the human would exercise.
[03:26] Okay. So, the second one is clarity. When something goes wrong or just generally, how can you reconstruct these workflows, um and how can you reconstruct what an agent actually is doing? Uh they you know, they work on
[03:38] open-ended tasks. Um their path of execution is of often um undeterministic and so, you know, observability becomes a very different problem from traditional software.
[03:55] And and finally, choice. Uh Uh you know, every every so every few months there is a new model, there is a new tool. Uh you want to embrace those and to be at the leading edge, but then how do you ensure that your governance structure actually
[04:08] uh allows you to, you know, continuously adopt new things without losing the benefit of governance? And how do you not get locked in to any particular model or tool?
[04:20] So So So So So these are the themes that we hear Databricks every day from customers, and it's become very top of mind for most of our customers. Um yeah, so here on this theme you can
[04:33] see how, you know, you're essentially kind of on the treadmill of, "Hey, what's the next best thing?" And teams wanting to adopt them, um but then there is a trade-off uh that comes to governance of how do you actually retain that kind of central governance
[04:46] approach? Okay, so we talked about those perils. Um let me walk you through kind of how we're thinking about solving those, and what are some specific capabilities that
[04:59] we just released that might help you in that. Let's start with control. Um control often starts with the question of uh authentication and
[05:12] authorization, right? And what we see is that you can take one path, which is essentially, "Hey, uh my agent is going to be um to have a a service principal, its own identity.
[05:26] We're going to lock it down uh so that it cannot, you know, do any harm. Um but then it's not very useful in many cases because uh it's kind of operates
[05:38] on the, you know, least common denominator. It doesn't have access to many systems. And so it's kind of not not getting us to kind of that AI Nirvana with all the automation we can
[05:50] do. Okay, so like the service principal approach might not be the best, right? The most common approach that we see is that well, okay, let's just let the agent inherent, you know, my identity
[06:03] operate on my behalf as the user and then it has access to everything I have access to. Is that good?
[06:15] It well, it definitely like allows the agent to access all these things that I have access to, you know, docs, confident confidential docs, PII data perhaps, you know, repositories. And we see like all these greens, right? So it does have the access.
[06:28] But it's is it really good for a you know, an agent to go and like if I have a permission to delete a GitHub repository to do that on my behalf, probably not. So that approach doesn't
[06:40] really work either. So what we think of Databricks is that access control for AI systems should evolve to be contextual.
[06:52] That means that the access control system should actually contain information about what kind of data the agent is trying to access to, under what context it does so in the in the
[07:04] sense that what is the task that is trying to solve and what were the previous actions in order for you to make an informed safe um access decision,
[07:16] but also ensure that agents are delivering on their promise of solving some difficult organizational tasks and workflows. How do we get there?
[07:29] Um So our approach is that because of this contextualized approach, access policies and access control has to be data aware.
[07:41] And so it's really doesn't make sense for AI governance to live in a silo and data governance to live in some other silo. So, what we're doing is we're unifying AI governance and data
[07:53] governance in one place under Unity Catalog. And what we've done is that we actually made all of these AI assets, whether these are um access to models, skills, agents, or
[08:07] MCPs, first-class securables within the within Unity Catalog. And then all that traffic that goes to these assets is is then intercepted and governed by Unity
[08:19] AI Gateway. So, let's see how it works kind of in practice. Some of the questions you might want to ask is like, what can an agent access? How can I restrict what the agent can
[08:31] do? What model can it use? Um so, we're here to help. So, the first thing we realized is that in many of these access control
[08:43] policies, what you want to do is be able to distinguish, is it David making an action or is it an agent operating on David's behalf? So, we've released a contextual
[08:56] um a context attribute that you can now use in your policies that where you can state that, "Hey, this access control applies when you know, someone is operating on my behalf." But it does not
[09:09] apply to me because I'm a human and I can perform that action. So, this is like a really strong instrument in your toolkit of now, you know, making that distinction and and making much more
[09:21] scoped permissions on your agents and what they can access. Um the second part is that in in in this new foundational
[09:34] um concept that we created, the the service policies, um you can also determine which exact tools do you allow agents to invoke when you call MCPs or
[09:47] tables um or or any of these um assets that they're trying to touch. So, you can attach the service policy on an MCP, and then you can express things like, "Hey, um within the let's say the GitHub
[10:01] MCP, um are you allow the agent is not allowed to perform things like push files into a repository. It can still read, it can still do many helpful things, but I want to prevent it from taking destructive
[10:14] actions." And now you can do that at the granularity of individual tool within each MCP. So, that's available to you as well.
[10:27] Okay, then the other thing we hear a lot is, "Hey, how I want to control which models can can I can my organization access and which they are not allowed to access?" Many have concerns over, you know, uh foreign
[10:39] models or just they have contractual obligations to use certain model family, and you want to apply those policies at scale. So, we launched support for ABAC grant policies
[10:52] with uh managed tags that allow you to control what are the approved models and what are the teams or organizations um that can have access to them. Um so, uh
[11:06] that that's a scalable way for you to be selective with what you enable in terms of the models that your teams can use. So, let's kind of like bring it all together of what are those capabilities
[11:19] that allow you to contextually select what agents can and cannot do. Um with the on behalf of context attribute, you can apply column masking, row filters capabilities
[11:32] uh in terms of content protection, you can use the service policies to restrict uh in terms of you know, allow, deny, or you know, human in the loop uh you know, allow request for MCPs.
[11:46] Um and then in terms of like pure access, you can have uh you can have uh control over which specific models your your organization can use. So, this kind of tool set gives you much richer
[11:59] uh set of um capabilities to control what AI can can and cannot access. Cool. So, let's move on to what we call clarity. And this is essentially if
[12:12] previously we talked about, you know, what agents can and cannot um access, here we will talk about what actually happened. Like, what is the uh you know, what is the observability that you can have over the AI usage in your
[12:25] organization. So, some things that you might want to answer in your teams like, what did my AI agent access? Uh what did it produce? How did it reach certain conclusion or decision? Um and
[12:38] uh something that we hear, you know, more and more as the most critical thing is how can I reason uh over ROI of AI and how can I reason about the costs uh that relate to it.
[12:51] So, let's talk a little bit about, you know, they refer to it as tokenomics. So, let's talk about tokenomics a little bit. Um we in the Unity AI Gateway uh recently
[13:04] enabled very rich capabilities of how you can track costs. And um that capability applies to any model um that you may want to use, any model or any tool. Uh you know, whether it's OpenAI models,
[13:17] Anthropic, even if you have some external capacity, you can bring it over to Unity AI Gateway. Uh but the main thing is that you get this tracking of uh all the usage at very high
[13:29] granularity based on what you want, whether it's like per user, per team, per line of business, you know, per the model itself. We give you kind of built-in um
[13:42] observability over all the all the usage, and you can set budgets on those same granularities. And And then um what we recently launched is actually enforce those
[13:55] budgets using hard spend caps, meaning that if someone gets to um over the threshold that you've set, then you can actually prevent them from consuming any more tokens. So, it's a very powerful capability and just works
[14:08] across the board with any any tool and any AI model that you're using. But moreover, you know, we think that over time the
[14:20] decision should not be binary, like do you are you over the budget or or not, and then you cannot use it the AI anymore. We think where you want to be at is to have some
[14:33] um contextual capability to determine like how um complex is a certain task and then route it to the most advanced frontier model if it is a um a complex task, but then
[14:48] route the easier, more trivial task to cheaper models because then you can get a much more balanced better ROI on your AI usage. And so, that's what we're deliver delivering in Unity AI Gateway, what we call
[15:01] intelligent routing, where you can have this kind of decision-making of how complex is the task and then having, you know, two models between which you route. One
[15:13] is the more advanced one, one is the more kind of cost-efficient one. >>
[15:26] all of these AI assets are now first-class uh securables in the Unity catalog. And just with that, you get a lot of observability out of the box. You get audit log um that gives you
[15:40] information of all the access patterns, uh all the API calls, uh everything that was called. You get lineage in both in terms of the upstream and the downstream um of what the agent accesses. So, like
[15:53] models, tables, uh you get this kind of full view of what uh systems that these agents touch. Um and then as I talked uh before, you get all these usage tables uh both in terms
[16:07] of tokens and in terms of requests and the actual spend dollars that you have over there. So, that's that's at your disposal. But one thing we kept hearing from you guys is that, "Hey, like it's really
[16:20] hard for me to reconstruct what an agent does because all this observability kind of lives in different places. And in most tools, you know, you have like MCP logs and you have like model logs and they're all disparate. And when agents
[16:34] talk to each other, then it's all in different places." So, what we've done is we actually unified all the tracing that happens across your AI assets in uh one Unity
[16:46] table. And then you can do some very powerful things with that. So, first, you can go and analyze uh a lot of things like what kind of questions my agent being asked. Like, so you can ask Genie, "Hey, you
[17:00] know, tell me what are the different tasks and categories that the agent is being used for." Uh you can perform evals using MLflow and improve over time the quality of your agent. And for
[17:13] security analysis, you you then turn uh Lake Watch uh and point at this table and perform very rich incident response or anomaly analysis workflows.
[17:27] So, so I hope you give it a try. Okay, let's get to our last pillar. So, we're building our governance for AI in Unity AI Gateway and Unity Catalog in an
[17:40] open and interoperable fashion and it it gives you a lot of you know, kind of powerful properties that come with that. So, first of all,
[17:52] all of these capabilities I'm talking about are backed by open source and open standards. So, Unity AI Gateway is actually open source under the MLflow distribution. So, you can even run it yourself if you
[18:04] would like to do so. Same for Unity Catalog. Um and then we're relying on open standards for everything that we do. We don't have any proprietary kind of you know, uh
[18:16] standards that we rely on. So, open telemetry for tracing. We recently partnered with a bunch of the industry partners to build a new protocol for sharing AI assets. So,
[18:28] you know, things like skills uh using open sharing MCPs and other open standards. So, uh this is just continuation of our kind of openness approach to the lakehouse.
[18:46] Now, in terms of kind of the selection, we talked about how you always want to be kind of at the frontier and have the ability to pick any, you know, any model, any tool. So, what we're doing there is that we have all the all the frontier models and all
[18:59] the most popular open source models hosted by Databricks. Uh so, you kind of see the list here. Um but in addition, you can also bring your own. So, if you have
[19:11] um you know, uh some inference capacity through Anthropic or OpenAI, you can use those and make sure that it goes through the Unified AI Gateway for governance. We're not opinionated in that regard.
[19:24] Moreover, to make it really easy for you to, um, use MCPs and access the tools that your agents need to be productive, we build managed MCPs
[19:36] for all of the most popular, um, SaaS tools that you might use, like SharePoint, G Drive, Atlassian, um, you know, Jira, Confluence, GitHub and and Glean.
[19:51] But moreover, something that we heard from you a lot is, you know, I I use these like disparate, uh, tools to, uh, for for AI for AI governance and for AI security, um, and how can I use those
[20:04] within the context of, um, you know, having, um, having Data Bridge being kind of the central governance place for that. So, we heard that feedback and we we
[20:16] partnered with many of the, um, best, uh, security providers, governance providers for AI, and those will have built-in first, uh, party integrations with Unified AI
[20:29] Gateway, so you can apply things like AI guardrails or identity attestation, uh, and things like that, uh, within the product, and it will all be integrated,
[20:42] um, seamlessly for for you. All right. Now, uh, I gave you a lot of theory, so I'll pass it on to Stefania to walk you through
[20:54] how it all comes together in a demo. Go ahead, Stefania. >> Thank you. Um, hi, everyone. Um, yeah, let's see how that works in action. So, um, in this, uh, demo, we'll first have, um, David run an agent without like any
[21:08] governance, so he'll do some coding like instructing the agent to update some code and then push it to get and then query some sensitive data. Then I'll show you how we can set up governance, so how you can get started
[21:21] if AI governance and then we we run the same example in a governed fashion so that you see how the agent gets restricted by what we put in place in Databricks. Okay, so first we have
[21:35] the demo here where so David is um updating a scoring threshold in config.yaml and then he wants to push it to GitHub, so the agent is doing the work.
[21:47] Um takes a minute. Um and then all of the all of the updates will be pushed to GitHub. So what we didn't see here
[21:59] and we'll show you in a second um is that it didn't just push um what he was updating, but in there's also an environment files where some like secrets were shared to GitHub,
[22:12] which is not really Oh. Um sorry. Yeah, so now in the next example >>
[22:30] We're actually so we're querying um uh Databricks get some data from the customers table um uh David has access to the customer table, so now the agent is doing a select on that table. We're going
[22:42] through the DBC core MCP and what we see is like we get all of the data including email address, phone numbers and so on. Um which is something we maybe also don't want the agent to have because
[22:54] they could in the next call like send it out via email because David can also send emails. Um so what are we doing to prevent all of that? Um, so, what I'm going to show you in a
[23:06] bit is how we configure uh the how we configure David to have access to Claude Opus through Databricks. So, then the local coding agent will be pointing to the Databricks
[23:18] endpoint for um for Claude. We set that up the MCP uh to go through um Databricks. I will also grant David access to the MCP, and then we'll make sure that we mask the
[23:30] sensitive data in cases of like agents are calling in on behalf of the user. So, here I'm in the Databricks workspace. I go to the AI Gateway. What you see here, we now have models and also MCPs.
[23:42] So, down here is the Claude Opus model. This is um uh an out-of-the-box model in under system.ai. I can open it. Um, you see there's various tabs. There's a permission tab
[23:55] um where all account users have been granted execute um on that one. Um, you can restrict that if you want, of course. And now we create a policy because we want to redact um
[24:07] we don't want secrets to be passed on. So, I create a custom policy for all account users. Um, I'm using a custom policy uh with an LLM as a judge. It should be applied to
[24:20] input and output guardrails, and basically deny any kind of prompt that includes um includes like security key. I've prepared a prompt here. So, um no sensitive data should be
[24:32] transmitted, no database connection strings, no API keys. Um I also give a couple of examples how these URLs may be or like these keys may be looking, um and create a policy. So,
[24:45] this is now applied to each and every call that go through that uh model endpoint. So, we've done that. Next up, we need to set up the GitHub MCP. Um
[24:57] So, we go to um the MCP tab. Um here we have various MCPs. We have the GitHub MCP. I'll open it. Um here we see in the description it
[25:10] says uh GitHub MCP no delete tool. What What does that mean? So, when you create the securable in UC, uh we're loading all the tools for you. You can select which ones should be made available through
[25:22] Databricks. Here we've made 42 out of 44 available. So, what we omitted are the ones where you can actually delete. We don't want any agent to like perform any destructive action, so you don't even
[25:34] provide them to them. That makes it very easy. Um and then now we need to Yeah, we've already granted David execute on this one. So, when the agent calls in on behalf of David, they can use um
[25:47] they can use uh this MCP. So, the last thing that we need to do is now um secure uh the PII data. So, we have um the marketing catalog. Here you see like how everything is
[25:59] neatly integrated is in the catalog explorer. Um so, we have the customers uh table uh with like email and phone number, which have like tags on it um because that's like PII data.
[26:12] Um and then we have um a policy um which masks PII. It's a regular RLS policy
[26:25] um as you know them. Um it's a column mask. You You get the SQL statement in the policy editor. Um it's a column mask applied to all account users um where we say, "Hey, if uh there's
[26:39] this context attribute um request is on behalf of equals true um perform the following action." Um we apply it to um the two columns email and phone number,
[26:53] when any um Yeah, when there's on behalf of we we just return a string instead of returning the actual data. Great. So, now we've set all of that up.
[27:06] What happened now, like locally, you can use UCode to configure your agent so that it's pointing to the model in Databricks, to the MCP in Databricks,
[27:18] and then all the traffic will be routed through Databricks to the MCPs, to the models, and you and all the access control is actually done and enforced in Databricks.
[27:30] So, if we go back to like David, who wants to perform the same actions again, we start with like querying the customer table. So, we're hitting again the DBC cool MCP. We're obviously
[27:44] calling in from an agent. Um Now, it's getting the data. And what you see is that email and phone phone numbers are now masked, so no more
[27:57] sensitive data um is being shared with the agent or can be queried by the agent. And now we rerun the scoring, the update to the config YAML as well, um trying to push the exact same um
[28:13] code that we wanted to push before. Um Now, we get an error because um we can't send like the sensitive data over um um
[28:25] over Yeah. So, you may ask yourself, how does this all work behind the scenes? It's actually relatively simple. In your local coding agent, you put the configuration
[28:38] um to which model they should be connecting, to which MCP service they would be connecting. You you them to Databricks. This works for Databricks managed models, the ones that we host, but also
[28:52] to any like external capacity you have with any of the model providers directly. You can still route all the traffic through Databricks. It's also not just like managed MCP servers, you can just bring in your MCP server into
[29:05] Databricks and route it through um uh just route the traffic through Databricks. So, then basically what happens is that the user authenticates, we carry on like the agent identity so
[29:18] that we know that it's a user acting on behalf like every audit um entry will always carry that information. All the traffic goes through the Unity AI Gateway. We do like um authorize against
[29:30] Unity Catalog like can David call the service? Can David call um the MCP? Does he have execute on those objects? And if we get a true back from
[29:42] like Unity Catalog, we then like execute the call. Call um external models MCP servers or the ones that we manage internally in Databricks. So, all the traffic is simply routed through gateway. You get like full traces of
[29:54] everything in Delta. You have heard that from David. It is basically all the tracing data is a Delta table again, which you can use to kind of gain some um insights as to what happened.
[30:08] Um yeah, and this is already the end of uh this presentation. So, what we've introduced here is like how we brought like all the AI assets into Unity Catalog. They're first-class citizen where you have like regular access
[30:21] control um that you have like with tables. You can apply direct grants, ABAC, everything that you have in Databricks. You get the same discovery that you have in Databricks
[30:33] and so that you have in Unity Catalog um by default. Um we introduced uh Unity AI Gateway, which is kind of your front door to a unity catalog for all agent interaction. And as David mentioned,
[30:46] this is not just for models, agents, MCPs, and skills. Like in Databricks, we're like very open. You can just manage any kind of like external agent, external models through Databricks in a
[31:00] very simple way. Um all of that will be in beta in like a couple of weeks. Um so we would really like you to give it a spin and also give us feedback. Um
[31:14] Yeah. So, thanks a lot for uh attending this presentation. >>

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.