Zero-Trust Agents: The Databricks Blueprint for Governing Least-Privilege AI

Summary

This video presents a zero-trust architectural blueprint for deploying AI agents in production on Databricks, explaining why roughly 80% of AI agents stall at the proof-of-concept stage due to concerns about over-permissioned access and unpredictable behavior causing real-world harms.
Three deployment scenarios are covered: internal productivity agents with least-privilege Unity Catalog access, agents connecting securely to external MCP servers using OAuth on-behalf-of authentication, and governed B2C agents serving external users through scoped tokens minted by Databricks Apps.
Unity Catalog serves as the governance engine, with identity-aware row-level filters, tool-level restrictions that travel with the agent, and full observability through inference tables and system tables that log all agent interactions for security audits.

Zero-Trust Agents: The Databricks Blueprint for Governing Least-Privilege AI

Most AI agents break when they leave the lab. this video shows you how to build ones that don't — using Databricks as your security-first production backbone.
You'll see battle-tested architectural patterns for three real-world scenarios: internal productivity agents with least-privilege access, agents that securely connect to external tools and MCP servers, and governed B2C agents serving non-Databricks users through external web apps.
At the core: Unity Catalog as your governance engine. We go deep on passing OAuth identity claims into UC functions for row-level data retrieval, applying identity-based filters within Vector Search, and enforcing tool-level restrictions that travel with your agent across every access pattern.
You'll leave with: a reusable architectural blueprint for production agent deployment, concrete techniques for identity-aware data access, and a governance model that scales without becoming a bottleneck.
Start shipping agents you can trust with Databricks.
Talk By: Alysson Souza, Product Specialist - Security, Databricks ; Ananya Roy, Snr Specialist Solution Architect, Databricks ;

Chapters

FAQs

Why do most AI agents stall at proof of concept and never reach production?

According to this video, approximately 80% of AI agents remain limited to proof-of-concept or pilot deployments because organizations lack confidence in deploying agents that may behave unpredictably or access more data than they should. Real-world examples include an agent that autonomously deleted an entire production database without human approval, causing severe consequences. The zero-trust approach addresses these concerns by constraining what agents can access and do, and by providing full auditability of every action.

How does Unity Catalog enforce least-privilege access for AI agents?

Unity Catalog allows administrators to define tool-level restrictions and identity-aware data access policies that travel with the agent regardless of how or where it is invoked. OAuth identity claims are passed into Unity Catalog functions so that each agent call retrieves only the data the authenticated user is permitted to see, applying row-level filters automatically. This video includes a demo showing least-privilege enforcement verified across multiple tool invocations.

How does OAuth on-behalf-of authentication work for Databricks agents connecting to external services?

OAuth on-behalf-of authentication allows Databricks Apps to mint scoped OAuth tokens that represent the end user's identity when an agent calls external tools or MCP servers. This means the agent acts on behalf of the authenticated user's permissions rather than a broad service account, limiting the blast radius if the agent behaves unexpectedly. The video shows this pattern applied for both internal tool access and B2C agent scenarios serving non-Databricks users through external web applications.

What observability tools are available for monitoring AI agent behavior in production?

Inference tables automatically log every request and response made by a Databricks agent, while system tables aggregate interaction data for analytics on access patterns, error rates, and usage volume. This video also demonstrates guardrail monitoring that detects policy violations in real time and can trigger alerts or blocks when an agent attempts unauthorized access. Combined, these tools give security and platform teams full visibility into agent behavior at production scale.

Full transcript

[00:07] Hello everyone. Thank you so much for joining at a very late. Um we are really thankful. So for this one just a quick disclaimer for any sessions you are
[00:19] doing then you would be getting some feedback requests. So if you can fill that up that will be really helpful for us. With that said um let us jump straight into the topic itself. So my
[00:31] name is Ana Roy. I am a AI specialist architect in data bricks and with me we have Allison who is our lead product specialist on everything security in
[00:43] data bricks and uh both of us today are going to cover a very important and um relevant topic today which is zero trust agent and how what is the databick's
[00:58] point of view on uh implementing least privilege AI in agentic world and hopefully all of these things will make sense to you. So yeah, let's dive into it.
[01:11] So let's take a situation that you all are building different types of agent. You have um spend months on building an agent um that is doing state-of-the-art
[01:23] work and it it's passing all of the evals. you're quite happy with it and now when it comes to moving into production um we are still skeptical we do not
[01:35] always think about okay it is the right time to move into production because there are a lot of different things that goes into our mind so some of the questions can be can I trust it what if um something goes wrong into production
[01:48] there are a lot of unknowns and these things are not only just occurring to us individually it is occurring everywhere and the main challenge which we are seeing across the entire globe when we
[02:02] work with all these customers is even though they have created state-of-the-art applications and AI applications 80% of the time it has still been limited with um either proof
[02:14] of concept or a pilot but uh production is still a challenge. Now what are those different challenges? Let's let's understand that part a little bit more. Um and before that um
[02:28] I'm just going to give you some real world examples of what's happening what we are seeing in the market today. So the first example is um this came out two months ago where
[02:42] one AI agent um so Pocket OS is a company where their agentic workflow has actually um identified a bug in their system and it identified that okay what
[02:55] should be the fix for this is to delete the entire production database without asking any human supervision. It did that and you can imagine what kind of effect that can be that company
[03:10] and that is not an isolated event. It happens over and over again as we see. So um Anthropic's own testing on a AI augmented vending machine showed that uh
[03:24] within one week customers were able to inject and trick the agent system and in first month it um to like um customers
[03:36] were able to get free drinks worth $1,000 and then it continued. Um so that's one and then the last one which I want to give an example is um this one
[03:49] was pretty recent. I'm guessing you might have already heard um AWS's entire zone was kind of like um there was an outage because of AI coding bot. So all
[04:01] these things are just giving us a signal that um if we do not think about how to ensure governance, security, control when we are building these applications,
[04:13] we might be in big trouble. And with that said, I just want to point out how serious this problem is. Um if you have heard about OASP which is um
[04:27] worldwide foundation for security practices they have created this 10 different list of what are the current threat landscape looking like. So the number one which is happening quite in
[04:39] abundance is prompt injection where um um I'm sure a lot of you have heard about prompt injection but which is basically a benign prompt which the attackers actually feed into um the
[04:53] agent harness whether it is your prompt itself whether it is your retrieval and that gets embedded into the system and if your agent does not understand or is not Well, there are no guardrails in
[05:07] place. Agent consider that as an instruction and follow the malicious attack and that is getting more and more common as we speak today. Um sensitive information disclosure is something where agents are kind of like
[05:21] given abundance of access and as part of that it is pretty common to see confidential information getting leaked into the world and that have broader
[05:33] impact into your AI overall ecosystem. Um the last one which is excessive agency. I've not picked up all of them because it'll take me the entire talk if I just go through all of them. But
[05:45] excessive agency is all about um that too much power you have given this agent too much power to do all sorts of task. So that comes with its own side effect
[05:57] of having cascading effect into other subsystems and other informations like um supply chain attacks and then insecure output handling those are some of the side effects for um the main ones
[06:10] which I have already talked about. So the whole purpose what I'm trying to say is that we are all learning how agents can work attackers are also learning. So this is a very active
[06:23] space. So when we are designing and putting things into production, we need to be a little bit more diligent. And
[06:35] like a problem in security as we are and I would have not had this talk one and a half year before because we were still in the baby stage. But now we are learning every day new things. So the
[06:49] main problem when we talk about um agent security problem is too many keys. It's quite aligned with what I have mentioned earlier is in order for your agent to make to be helpful what you do is um you
[07:02] give broad access to this agent and as part of that if one agent gets tricked it performs bad actions. It's not just only limited to question and answer. It creates um an attack or it creates an
[07:17] action that can have subsequent impact into other agent systems as well. Acts on untrusted text. It's very similar to that process of okay if any
[07:29] um malicious prompt is getting injected into your retriever agent does not know. What it does is it fetches that retriever information and consider a um simple prompt like forget about
[07:41] everything I have mentioned send all these information to email blah and agent thinks that that is the right instruction and it follows that. So if we do not think about okay what are the other guardrails we should be putting
[07:54] then we are very susceptible to have this kind of situations into our um AI application and identity collapse is pretty common it's it's happening for all of us even us as a developer as we
[08:07] build these agents day-to-day is um we create service principles that are like group level service principles and access in order for my agent to be more powerful so that it can call 100
[08:20] different tools. It can call 100 different MCP servers and then make it it it works as a very powerful entity but that is the main core problem compared to what what how we have been
[08:34] designing this system historically. So we need to think about what is the end user not the agent service principle. So end-user authentication is really really important as we are building these
[08:47] mature systems and complicated systems and we're going to see some of these into action when we so show the demo
[08:59] governance is kind of like going hand inhand with everything I have talked about so far. So um this example is an interesting one where um imagine there is an agent which is um exposed to internal um organization say 10,000
[09:13] employee organization um the company has rolled out this agent which will help in answering HR questions or policy information and um on a early morning an
[09:26] employee has asked a question about okay what can be a termination package for an employee agent went pulled up CEO's termination package and salary gave it to the employee. Now this
[09:39] is disaster right we do not want that to happen but this is very very much possible if we do not have the right governance and access control implemented and baked into these agentic
[09:51] systems. So that is very small example of what can happen if things can go wrong. Having right traceability, having the right data access and governance and flexibility so that you
[10:04] can switch between different agents is is going to be quite key when you are building these systems. With that we introduce zero trust for agents where we are going with the same
[10:17] concept of never trust and always verify and verify the user the end entity not the agent itself and verify where the data lives. So let's understand that in
[10:29] practice with three different scenarios one after the other and that's like our own common day-to-day work which we do today. So first one is internal productivity
[10:43] agent. So that is your standard coding agent um your cloud code your codeex however and I'm guessing majority of you are using some form of coding assistant
[10:55] or coding agents in your day-to-day work and the way currently it is working is it is getting rolled out at an enterprise level and everyone is kind of like doing their own development in a siloed laptop. So um that's the main
[11:08] problem of having a centralized governance. So what is data brick's point of view here? We going to see this with a demo and external tools and MCPS are basically you have done your
[11:21] development. Now you have built an agent for your enterprise. You put that into production which is doing 100 different task. It's f fetching information from different MCP servers, calling different agents and doing all sorts of stuff. and
[11:34] how do I ensure that it is governed security controlled and within its right parameter. And last but not the least, this one is um B2C agent which is where
[11:48] most of you if you are going to build your end user application, you would put it in front of your end customers who are who are completely oblivious of the fact of what environment you're using,
[12:01] what platform you're using and what not. What they want is some answers or some work to be done. So how do we ensure um governance and security and access
[12:13] control for B2C type of agent? So all of these different things goes with the same underlying concept but the way to implement it is a little bit different. So let's see that for the first one which is internal productivity agent.
[12:28] So based on our understanding the typical architecture looks like your developers different teams and developers are in um starting with their development in their own laptop and then
[12:40] they use these different productivity agent harnesses. Um so it can be your cloud agent or your codeex open code pi so on and so forth whatever of your
[12:52] choice. And then how can I have at an enterprise level govern all of this so that everything is trusted. So for that datab bricks has a
[13:04] capability of AI gateway. What it does is it automatically links all of your um coding agents with uh coding agent traffics with Unity AI gateway and what
[13:16] you get out of this traces and logs. every execution get traced and logged inside datab bricks. That's number one. Number two, rate limiting. It's pretty important and
[13:29] almost a given for budget control. So you need to have right level of rate limiting at a user level, at a group level, what not. Number three, access control. It is the whole point of the
[13:41] talk. How would I ensure that access is controlled properly? Number four, observability. Once I have got all of these different traces, how do I ensure that it is
[13:53] getting worked um it is working as expected? Um the evaluations are running as proper and the whole suite of observability which comes into practice. Policy and guardrails um if something
[14:08] goes wrong I need to be in a position so that I can fix it before it happens. And that applies not only for my custom agent, that applies for my coding agents as well. How do I do that? If I use
[14:20] Unity AI gateway, I get all of those things almost by default for free. And um this is where the difference lies of building a siloed individual application
[14:33] that is not scalable versus an application that is governed, managed, and scaled to millions of users. Right? So with that said, let me show you how it looks like. It's a very small one.
[14:47] So what I have here is my terminal. And imagine I'm a developer. I am using lots of different types of coding agents at the same time. And I want to test it
[15:00] out. Now if you have used it on a daily basis, you know this is how your terminal should look like. So nothing fancy on that. But let's see what happens. So in my different terminal,
[15:14] I'm using something called Ude. So before starting there, what is U code? Um Ukode is um an automated way. So your all of your coding um agents will be
[15:27] wrapped around with Darts AI gateway. it will be using um the model serving endpoint whatever of your choice and then you will get all of the things which I have earlier mentioned about
[15:39] rate limit governance access control by default along with authorization management so you do not need to maintain your authorization all the token refresh everything will be done automatically sorry what happened this
[15:51] one
[16:05] terminal I am opening up claude and in the third one and remember that I didn't have all of these things set up in my local terminal at all I can if I am using but u code automatically does that
[16:17] if it's not already present the third one is open code which is a popular open source coding agent framework and copilot um I can add as much as I want um but the idea is like from developer
[16:30] perspective nothing changes and you can start with your models your work as expected. So for example for claude code I have chosen a model like sonnet 4.6 and for um codex I have used gpt 5.4
[16:46] mini and I have selected this model to show something which I'm going to come into in a moment. For open code datab bricks also provides capability for open source models. So for example here I'm
[16:58] using Neimotron. Um there are a couple of different options which you can choose from and copilot is very similar. You can choose any model of your choice and then you can start your work as expected. Like for example you can say
[17:11] write a Python program to perform Fibonacci series and you can actually continue your work and then it will do um its work. you would not see much of a
[17:23] change.
[17:37] task you have mentioned. So nothing fancy here but the main difference is everything is happening under the hood. So all of your execution logs are getting traced inside datab bricks. you
[17:51] have the right governance and control. So for example here I'm going into claude 4.6. If you remember our claude session with 4.6 was there. There are different um guardrails I have
[18:03] implemented. So I've implemented PII detection guardrail um which is going to look at any PII information and it can block. So this is the model which is getting used in my coding agents automatically. I have not done anything
[18:16] extra. The metrics is actually showing okay how all of the informations are getting tracked and fetched and we have got a very beautiful dashboard which shows all those things in detail. I'm
[18:28] not going to go through on all the details of gateway but the same um logic I have applied for 5.4 as well. Um very similar logic which I have done for sonnet 4.6. You can see the matrix. You
[18:41] can also see the different types of the guardrails which I have implemented so on and so forth. So with this one, what I want to show you is how all of these
[18:54] things are actually working. So I go back to my terminal and I say um I've just given up a very basic uh PII information and let's see what happens.
[19:06] So for codeex GPT 5.4 four is getting executed and what you would see is that the guardrail gets invoked um as part of this. So that ensures that all of my
[19:19] enterprise level guardrails are getting um enforced with every developers. Think about every developer um your security team or your infrastructure team has done it for once and then that is
[19:32] getting implemented across all of the enterprises or um users who are using it and the same thing you are seeing for cloud code so on and so forth. So that's one thing. You can also see that the
[19:46] inference tables has all of the coding agent traces gets logged into Unity catalog and Allison will show that in more detail of every once every and this
[19:59] is the blueprint of all the entire agent execution. So if everything is logged the potential is kind of unlimited of what we can do whether it is your eval whether it is your security thread detection so on and so forth. So with
[20:13] that said there are few more things but um we can cover that at a later point of time. I want to go into the next topic. So this is all about internal coding agents but let's see what happens for
[20:27] the second time second um use case which is our custom agents which can be used and that is the typical workflow you use is that you create your agent that is using bunch of different MCP servers
[20:39] bunch of different sub agents multi- aentic system so how can I ensure that it is also governed in the same principle as the first one so what happens is
[20:52] first your employees authenticate and then um we have a concept called on behalf of authentication. So which is basically authentication pass through. So what happens when your agent executes
[21:05] your sub agent executes the agent um like the core agent itself no matter which type of server you use services you use MCP whether your SQL your unity
[21:17] catalog function all of them are first of all governed and also end user permissions are getting passed on not your agent service principle if you want you can create a broader group level
[21:30] service principle but that's the point of the whole um conversation that we want to have enduser access control passed on. So employees authenticate their identity flows through the agent
[21:43] then AI gateway centralizes the routing and whatnot which I have already mentioned. Um agent also enforces um the unity catalog permission control and
[21:55] then we also have the monitoring and audit on every request and that is maybe scratching the surface. I'm just showing only four but there are hundreds of different other capabilities that comes when you build agent with um the right
[22:10] framework in mind and um this is at a individual tool or MCP level uh conference like um architecture but when it comes to your
[22:23] say external MCP things are slowly starting to get a little bit more complicated that I'm not just only using individual MCP I'm using somebody else's MCP server to my agent then say for
[22:36] example a Jira MCP Slack MCP they have their own way of performing um access controls and whatnot how can I ensure that is also following the same principle as my other MCP. So typically
[22:50] what you would do here in datab bricks is um for external MCP you would be creating a unity catalog connection and that connection creates a wrapper and it um of your external MCP and it follows
[23:04] the same protocol as your standard MCP. um at the fret phase value you will not be able to differentiate between a managed MCP server like Genie or um
[23:16] vector search index versus an um external MCP because Unity catalog created a wrapper on top of it and it follows the same principle as your other MCP servers. So that's how we ensures
[23:29] that the same unity catalog governance and access control is enforced for your external systems as well. So um just to put things into
[23:41] perspective um as I mentioned identity and access we are mention uh doing it through OBO. U grants are enforcing least privilege access and um we would encourage you to
[23:55] not use service principle broad level but use OBO cost and control and everything else is handled by AI gateway. We'll not dive deep into AI gateway because this is not a AI gateway talk. AI gateway has hundreds of
[24:08] functionalities. We have had keynotes along with product sessions. Highly encourage you to look at that if you are interested. Um for this custom agent architecture what I have done for this demo is I have created an app um that is
[24:23] hosting that is hosting as a front end very simple app nothing fancy and the back end is a supervisor agent um which has capability of accessing genie space and UC function. So this is a
[24:36] conversational agent I have created for an education domain which also has web search capability. So I've used u.com's external MCP server and then I have created this agent and then I have
[24:48] hosted it in databicks apps. Um I have wrapped around the entire thing into Unity AI gateway so that it is governed with everything I have talked about and automatically all of the agent
[25:01] conversations are getting logged into Unity catalog and it is getting um it is also getting all the benefits which MLflow offers today as part of um Unity catalog logging and tracing.
[25:15] So with that said, these are all about inference tables. But let me show you the demo a little bit more. So let me see if I can show you. So this
[25:29] agent um is the one which I have created. It's a very simple agent which is um a react application at the front end. And what you see here is that um this multi-agent architecture I have
[25:43] created. So I have got basically all of the permissions. So when I ask what are your capabilities, it says I can actually perform genie structured data fetching um and then web search and
[25:56] in-depth research blah blah blah a lot of different capability and I can ask can you search the latest recommendation so we both are from Australia so that's why we wanted to understand okay what are the best restaurants we can go in
[26:09] San Fran. So that is what I was asking and then what my agent has done is it has gone and then fetched called my MCP server and then fetched all those
[26:21] information and give me a nice um kind of like recommendation of where we would be able to go. So that is fine that is good and then I can go into detail about
[26:33] all the different architectures but the interesting part is how O is getting maintained. So the app service principle is there but it is using my authentication as in Ana's authentication and it has got all the
[26:47] cap access to Genie MCP MCP functions whatnot. So I would be able to access everything but as part of that I have
[26:59] asked Allison um to test this function and he did that. So let me show you before uh what Alishun's um
[27:12] screen is looking like. So when he tested this agent um I have given him some permission but not all. And when he tested he asked the same question to the agent and what he gets this is something
[27:26] where um this is just we are showing that Allison has kind of like logged in here and when he asks the same question what he gets is a little bit different than
[27:38] what I see. So he can access the genie space which I have given access. He can also have access to the um UC functions which I have given access to but he cannot do web search. Right? So he asked
[27:52] the question specifically about okay can you perform web search and the agent has come back saying no you cannot. So this emphasizes the fact that okay if we have the right on behalf of authentication
[28:04] and mechanisms in place we can make the same agent behave differently to my different users. And um just to expand that part a little bit more. When I looked at the log, I see um that my web
[28:19] search MCP server has failed and it is failing with 43 error which is because um Allison does not have access to that particular
[28:31] yes it does not have particular access to my MCP server. So it fails and this entire log is also getting stored into my unity catalog so that I can have a look. So every interactions is getting
[28:43] tracked. So, um I think what I would say is um sorry my screenshot I was too lazy to kind of fix it but um yeah so the point
[28:57] here is that um with the second entry of like okay how can I build my custom agent is if I have the right OBO um in place then I would be able to um ensure
[29:12] that the right access control and policies are properly behaving for all of the users. Now we have covered two things. Number one internal productivity agent. Number two is our custom agents
[29:25] which we have built. And now I will hand it over to Ali to talk about the next part which is B2C agent and lakewatch. So
[29:37] thanks. >> Thank you. >> Uh hi everyone again. I'm Alison Sza. I'm part of the security team at data bricks. uh and like she said, we're going to focus on the three different patterns. But just before we do that, I wanted to do a parallel and talk a lot a
[29:50] little bit about the use of logs as important signals from a security perspective. Now, Anonia showed you how the uni uh the Unity AI gateway can be used to log traces and log interactions
[30:03] between users and an agent. uh and that is valuable information that you can use to understand the performance of your system, which tools are being used, but it's also very important when you're looking at it from a security
[30:15] perspective. Okay. Um now you if you attended the the keynotes yesterday and today uh you saw
[30:27] two things. First you saw that we announced lakewatch a new service running on top of data bricks that allows you to kind of build security detections and anal an analyze security data but you also heard Ali saying uh many times and many people repeating it
[30:40] that uh AI doesn't have an intelligence problem they are very smart already but they miss context and particularly on security space what we see as a challenge is that currently a lot of
[30:52] security data is sitting in silos each tool produces its own and it's very hard to have a comprehensive view of across the state, right, of everything that is being generated. And like I said, inference tables is a good example of that because that's valuable data. It's
[31:05] how your agents are behaving. It's what tools are calling. It's who who's interacting with them, which triggers are being called. So the idea here, what I wanted to show is how an open security lake can be a foundation to ingest all
[31:17] this data, including these interactions between agents and your users, agents and other agents, agents and tools. Uh so you can build detections on top of it. I did prepare a quick demo. Uh let's see if we can go through it.
[31:30] Um so I just have access my workspace. Sorry one second. And I created in this
[31:42] case similar to what Ana has shown. I created an AI gateway and I have configured it with LLM in this case us 4.6 I believe sonet 4.6. I configured inference tables and my guard rails. In
[31:56] this case, my guard rails are looking just for a jailbreak and prompt injection attacks. And for this particular demo, because I wanted to do different kind of requests, I I configured it for log only mode. Now,
[32:08] what this means is that this information is being logged in my inference tables and my uh when something triggers I get some context around what is that trigger that so I can see the full request, the tools and everything that was called.
[32:21] You can see the sample data just like Ana showed in my inference payload here. Now with Lakewatch I can access this information. So if you have lakewatch enabled it appears on like the app switcher at the top right corner and
[32:34] essentially what I can do is um this is the lakewatch UI and essentially I can do is I can point it to my data source which in this case is my inference table that was created before. I had already configured some data sources there. But
[32:48] here what I'm going to do is I'm going to say hey now look at this data too because the idea is that the agents in lakewatch they're really powered towards the investigation and the threat analysis and the building of security
[33:00] detections. So I'm going to give it a name just like to identify this source within the system. So this is my guardrail data and I'm going to point it to this to the table. So I'm going to
[33:12] search for my guardrail inference table here. The model it was using for the guardrail is a GPT5 nano and essentially lakewatch detects the schema and understands the data for me. So this is what it's doing at this stage and it
[33:25] will add some metadata. This is how we can keep track of any new records that are coming on the system. It has metadata like ingested time and analyze time and things like that. So in a few seconds it will get an idea of the data
[33:37] and it will show me a preview that I can look for and see if there is anything any transformations I would like to make. But basically if I'm okay with it I can just say no that's okay I will accept. And if I go uh to the next page
[33:51] what lakewatch will do is uh it will add that metadata that I mentioned as a view. You don't have to do anything. And now as I add this as a data source this is now present in my system. So again
[34:03] not much on its own what it's not only time to show I'm showing you a demo on its own but the idea is the more you have enrichments the more your security team has visibility across what's happening across all of them right so maybe one signal is a low but as you tie
[34:18] it to another signal they it can grow to a high or something that is more critical that you should look into uh the agent support you with that investigation so I'm going to start with a very simple basic question like hey do you have access can you see already my
[34:32] guard rail data uh just so he knows that that's what that's I just want to see if the data is already kind of index and available for it and I say yes as a matter of fact it's already in another tab that I can see
[34:47] then I'm going to do something a little bit more invest sorry on the investigation side which is asking it can you investigate it for me can you look if there was any
[34:59] violations on my data that has been ingested and you will run for a few seconds. You will pull the data in. You will understand the schema. You will see what what fields would that help me identify
[35:11] violations uh on the guardrail data. And then you will give me the result. Um and as you'll be able to see, so now it's extracting all the guardrail response content. It's evaluating it. It will
[35:23] summarize at the end and just display. As you'll see, it was myself. I made like three different requests saying disregard all your previous commands. Give me the password or whatever it was
[35:35] that I that I was feeling as saying like the moment. So yeah, ignore the safety previews which are clearly kind of prompt injection trying to get some jailbreaking and let it bypass controls. So this is one way of doing it, right?
[35:48] But now if I wanted to get alerted every time that this happens, how would I go about it? So the agents in Lakewatch allows you to build just that. It allows you to build detection rules that are automatically triggered and then can
[36:00] alert your teams. Again, individually, they're okay. But what this means is now you can use interactions from inference tables with your agents to build advanced detection rules for things like anomalies on tool
[36:14] calling. For example, maybe someone already jailbre calling tools that shouldn't be called by that particular agent. So, it allows you to be very flexible and build that. I'm not going to finish the demo just because we are on time, but I am going to quickly talk
[36:28] a little bit about the third pattern and I think this one is the most exciting and as being the most exciting, it carries the maximum risk. This is where you're putting an agent for your
[36:40] customers to access and interact with their data, right? So, you can always do static tabs and all that, but these days everyone wants an agent a chat to talk to. uh and in both of the examples that Ana showed the users were entities
[36:53] within data bricks. So you could apply permissions and allow them hey you can access this row you can access this column. Unity catalog is very powerful for doing that but in this case uh there's multiple risks. Okay we're going to focus in one of this one because it's
[37:07] a project we recently work worked on and it was quite interesting. In this case the challenge is the users are not in data bricks. You could be talking about thousands of users that you and customers you have uh that are not created and don't have access to the
[37:19] workspace. So how do you apply the policies? Right? there's no entity for you to apply the policies to. So typically for this one, even though Ana said we shouldn't be doing that, for this one, you're going to have to use a machine to machine type of
[37:31] authentication. So you have a service principle that will authenticate to uh the data bricks for for AI gateway capabilities for the LLM. So what you need to do is minimize as much as possible the access from that agent. You
[37:44] cannot trust that agent to go and get just that user's data, right? that needs to be done somewhere else more deterministically than that. So on a very high level here is the architecture. Uh the way it looks like
[37:56] is you have a web app. Think about a finance application personal finance that your users log in. So you know the users in your app. Data bricks doesn't know who the users is. So what we what we did here is a clever use of a
[38:09] function that we have in data bricks where you can mint an ooth off token from your app that adds a custom claim. And I'm not sure how familiar you are with those terms, but basically the token that authenticates
[38:22] identifies who the user is. There is a unique uh string attached to it. And I'm show you an example of how it looks like. But basically what this allows us to do in the data brick side is even though we
[38:35] cannot apply UC permissions directly to the user when the call comes in, we can extract that identity and using a unity catalog function, I can apply my filter. So I can only extract that data. So I
[38:47] don't need to give my service principle full access to all the tables. So it it can execute everything and trust that the user is not going to make a prompt like give me the expenses of user B. Uh so this is a way of you of how you can
[39:00] do that. I'll show you the code very briefly but essentially uh the way you do that is when you're requesting the token think about your application you're requesting you know the user it's logged into your app and now you're
[39:12] requesting the token to make the call to the datab bricks kind of AI gateway for the response so at the end in red what I have added is a user ID field now on the data brick side again instead of giving
[39:24] it access to the the whole table what I create is Unity catalog functions and the functions are very specific specific. So for example, if my agent needs to be able to search transactions for that user, I have a very specific
[39:37] rule where I'm selecting all the fields that I want, but when my wear clause on my filter, I'm not taking any inputs from the agent there. I'm always filtering based on the ID of the user that is requesting it. This is powerful
[39:50] even outside of uh the agent conversation. So we have customers who built their own applications and use data bricks under the covers to process and analyze their customers data. So this is a mechanism that allows you to
[40:02] identify the users externally even though they are not existing data bricks users in the ecosystem. Uh unfortunately we ran out of time. I did run a demo. believe this gets posted
[40:14] but I just wanted to do uh instead of the recap just a few things to consider right the first thing make sure that governance is centralized it's important people are using different tools every
[40:26] day so making sure you have a central place to organize and apply your controls and governance and monitoring uh rate limiting everything else that we talked about enforce list privilege like ana said if possible always go to the
[40:39] identity of the user who's accessing the systems But when it gets interesting and it gets the most challenging kind of use cases, minimize as much as possible the access and don't trust the agent to make the the critical decisions for you. Uh
[40:53] and finally maintain the architecture flexible like this is changing I don't know so often that probably by the time you watch the replay of this video there's going to be a different way of doing it. So the idea of having plug and
[41:05] play and the idea of modularizing it so you can switch different parts but keeping that central uh governance applied uh is key. Uh thank you very much for your time. I sorry I had to
[41:17] rush at the end uh but hopefully you it was uh useful for you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.