Sponsored by: Immuta | Agentic Impact to Secure Data Provisioning

Summary

Secure data provisioning: Balance security, compliance, and innovation while provisioning data at scale with AI Agents as decision makers.
Data governance evolution: Learn how organizations shift from single gatekeepers to scalable, policy-driven data access frameworks.
Immuta partnership: Explore how Immuta and the Databricks Data and AI platform secure AI initiatives through effective data governance.

FAQs

How do you balance security and data accessibility for AI agents?

In this video, Immuta VP Matthew Vogt discusses how organizations can design data provisioning policies that secure AI initiatives while enabling agents to access the data they need. The approach emphasizes governance frameworks that prevent unauthorized access while promoting innovation and data-driven decision making.

What are the challenges of scaling data governance?

This video explores how traditional data governance, where a single person manually provisions access, doesn't scale. Modern solutions require automated, policy-driven approaches that leverage the Databricks Data and AI platform to manage access at enterprise scale without creating bottlenecks.

Who can access our data and systems?

This video from Immuta highlights the challenge: traditional ticket-based access control breaks under the scale of human and AI data consumers. Modern solutions require policy-driven approaches through the Databricks Data and AI platform and partners like Immuta to automatically manage access based on roles and permissions, ensuring only authorized users and agents can retrieve specific data.

How can AI agents access data using the permissions of the requesting user?

This video explores how organizations can design data provisioning policies where AI agents operate within the security context of their requesting user. By integrating governance frameworks directly into the data access layer of Databricks Data and AI platform, agents can retrieve data autonomously while respecting user permissions, enabling faster decisions without manual approval bottlenecks.

How can companies govern data for AI applications?

This video demonstrates strategies for implementing effective and secure data provisioning when AI agents are decision makers. The approach integrates governance controls directly into the data access layer, ensuring compliance while empowering agents to operate autonomously.

Full transcript

All right, good afternoon. Uh, day two I think of the data brick summit. Thanks for grabbing some time with us this afternoon. So I believe we have to show this for everything even though we may not be making future forward statements. This is uh covers everything I'm going to be talking about as well as want to remind everybody to please fill out the survey afterwards. That obviously helps data bricks understand what content's being helpful for you but also it helps us uh and helps me as a presenter understand as well like are we presenting content uh that you are finding helpful and useful.
So, so once upon a time I used to be responsible for within a school their financial student uh data and so that meant I was responsible for running ad hoc reports that means I was responsible for creating pre-anned reports uh writing all the custom SQL that people would need and so I was I had to know all of the regulatory responsibilities for the school for who should get access to data if someone was requesting access.
I had to know uh so FURPA is a a regulation in the US that governs access to student data and I had to know was someone requesting access should they get access based on this law and regulation. So essentially I was the gatekeeper for provisioning data for organizations which I don't know if you know this I am I don't scale and so I was a single person provisioning that access. So in that in that sense it was actually very easy.
This access of provisioning access to data was very very simple and could be managed. So I'm going to actually jump into the wayback machine and we're going to talk about how we got access to data, how we've come along so far and really looking forward to today's world of creating agents for access. How do we how do we get here and what are some strategies for managing that access? So, like I said, we're going to go way, way, way, way back.
So, about 42,000 years ago, we started putting pictures on walls. That made things pretty secure because you had to have access to the cave to get access to the data. So, in that one sense, again, it made it pretty secure, pretty safe, but it didn't really make it very sharable and very portable. People really couldn't learn from it. And so, because you had to go to the data that might be, you know, 50,000 miles away.
And then about you know another 35 40,000 years beyond that it's a pretty big stretch we invented writing right so kuniform uh started to make data portable we started putting them on tablets we started writing things down on smaller more easily consumed items and what that led to is then the collection of these smaller forms of portable data into the first library which is about about three about a thousand years I guess after really the first tablet And what's interesting about especially these first libraries is they were putting in place and putting in their kind of data asset repository a lot of the same things that we are putting in our data asset repositories transactional information uh inventories things that maybe even considered sensitive data.
So we started cataloging this data, creating records of where does this data exist and even sharing that catalog around of you know where can I go to find this type of information within my kingdom right so we have you know 50,000 years ago we had writing then about 40,000 years ago or 6,000 years ago we started uh putting them on tablets and sharing it and so leap forward now we're moving faster and faster and faster towards how we're sharing data how we're accessing data and how we're provisioning access to that data.
And so we started with the cloud about 2002 or so it's about when S3 really came into prominence and about 15 years later uh Zamakani started this paper around data mesh and federated architecture and federated provisioning access to data and then again we've been moving faster and faster to invent systems of access to data. the challenge we're running into now. So previously, again back to when I was responsible for data, I was one person provisioning access to data.
We're not really in that world anymore. So we need to understand how we can safely provision access to data both to the scale of humans that are wanting access to data and in the future very very near future of agents that are needing access to data. Because what we're looking at right now is typically on the scale as we have an explosion of data consumers in our organizations and a very small number of people actually building data products and approving data products.
And so actually a a quick a quick request. So who here in the world of data governance and people approving access to data who has like a data governor as a a titled role in the organization? That's like like that's pretty typical, right? So it's not really a role in organization. It's typically like an other duties as assigned within organizations. And so what's happening is it's we have these other duties as assigned typically a small number of people managing access requests for a growing number of people.
So with the advent of things like claude or chad GPT where people are generally giving access within uh to kind of non-technical resources we have now this explosion of human data consumers but without really this idea of growing um kind of this growing data governors or growing data stewards and so what's happened is we have this this concept of data provisioning that so you might have then I'll tell another raise of hands who uses like a ticket system to manage access to data like who's going to like service now or Jira or something else like that right so actually keep keep your hands raised so who who has who likes it okay that's kind of the general reaction that I'm getting every single time right so we're in this world that I kind of call ticket based access control it's really it's it's really breaking under the weight of human access to data and what I really want people to start thinking of if that system is broken for a human access to data.
What's going to happen when agents start also asking for data at this massive scale? Because what we already have is this explosion of data consumers, like I talked about, putting this massive pressure downward into people whose job it isn't already to approve that access to data, which then puts even more pressure into typically centralized IT systems to provide hopefully scalable systems to grow the business. And so kind of like what we think about at Ammuda is this idea of the plight of the data governor.
Again, the data governor isn't really a job title. It's typically an other duty as assigned or data governor or data steward, you know, people who have domain knowledge around data and can approve access. And so we force these people today into a very IT ccentric again ticketing based system where people are doing manual work to grant access to data even after that ticket system has been processed. So we have a customer today.
So they're in the Nordics. They are a pharmaceutical customer. They make a very popular weight loss drug. In just one business unit of R&D, they're people who I guess who were assigned to do governance. They were fielding 200,000 data requests a year. So it's for a very small team of people. That's 800 data requests every single day. Obviously, like that's just humans getting access to data. That's not going to scale.
And so our goal as a company is to really think about this. How do we how do we remove the toil? How do we scale this idea of approvers to access to data? And so we think of this kind of in three distinct eras. So era one, not that long ago, this idea of making this idea of so like who's heard of like the term of birthright access to data? And so it's essentially it's because of who you are, you get access to this data, right?
So you're kind of born into this access in an organization. And so we needed to scale that even through this idea of like no code policy builders. So you're not writing this in SQL. So non-technical users like governance people or data stewards should be able to write policies of who gets access to data again just by default by way of who people are. And kind of where we are today is this second era of how do we scale this through that request workflow, right?
Because there's kind of two workflows of how you get access to data. One is birthright. You get data because of who you are. The second is a request system. Maybe I don't get it by default. So I go searching a data marketplace, a data product marketplace and I find data and I have to request it and that goes through a flow. So how do we make that better? So you know we at are really focused today on this scaling the governor through this workflow request by being that last mile of provisioning.
So having a request system where within that request system that actually goes and into the data platform into data bricks and says to unit catalog based on this decision give someone access right so you kind of remove that ticket based system so that's where we are today but we have this very quickly oncoming era 3 that's kind of where I'm generally going to go right here you can probably guess exactly where it's going to go uh but we'll get there in a second so with this idea of the reason more consumers your your consumer constitu constituency is exploding is we have so many more AI applications that are giving non-technical people non kind of SQL heroes or SQL wizards nonmats access to all your data right so we have the AIBI we have like the products like data bricks genie you know we have people building rag applications through um you know through different vector stores uh and so actually data bricks actually put out a survey last year that's about 60% of organizations The first AI application was something around the rag architecture.
But what this really meant again was non-technical people having direct access to all this data. But we're going from this idea of explosion to consumers. We believe in the next era of agents that is going to far outpace the number of consumers, actual humans that are accessing data, requesting access to data to agents. And we think that's going to be 10x to start and then it's 100x and that's a thousand agents, right? these nonhuman identities requesting access to data.
So we have this concept of ticket or agents essentially acting like other humans going through the same process of maybe they don't have access to data through birthright. They don't it's not scoped to the agent and so they want to try to pull in other data to make extra insights. They're going to have to go out and search for and find and request for access for data. And this ticket based access control is just not going to work.
So, we've got to move from this idea of a ticket based system where it's manual policy creation. It's manual provisioning via workflow. Again, humans in the loop everywhere. Every time you have a human in the loop, that slows things down to moving into more of a native marketplace. So, there are lots of vendors, you'll probably talk to a lot of them here, that have this concept of data product marketplaces. Whether that's an existing catalog you might have today or you're looking at something new.
You know, how can we take this idea of a data product marketplace where you're just searching for data and asking for data and now extend that into actually provisioning access once that decision is made from the data product marketplace. And then we need to think about is that foundation you're building from this uh native marketplace is that scalable for when we bring on this idea of agentic access to data. And so that's this idea of a governance agent.
Can they actually make the same decisions a human would and how much do you trust it? And I think kind of my my hypothesis is a lot of people aren't going to trust immediately a governance agent approving access to data. All right. So who who like who's going to trust that today? Very few people. And so what you need to be able to do what we think is you need to be able to start slow, dip your toe in the water and say classify the request somehow to say maybe an access request might be low risk or it might be high risk.
Would you trust an agent to make a low risk? That's you might have a classification of data that's public nonsensitive you know all the way up to like highly classified or sensitive. Can you trust agents to make access request decisions for that very low risk you know maybe public data? You just need to you need to have an audit trail over someone access uh asking for access and someone granting access. And so as you probably have already guessed this idea of era 3 is how can we leverage at scale this idea of a governance agent?
And so this is where we're focused today and uh on from a future perspective how can we dip our toe into what can a governance agents do for us today? And when people talk about agents and even like generative AI or AI in the enterprise, there's a couple ways people dip their toe. And one is kind of AI or agent assisted human interaction that can an AI make a recommendation for me to take an action and can that help me scale or do I actually have the agent make that decision for me?
And so at Ammuda what we want to look take a look at is how does this engage obviously we're at the database conference and how do we engage the unity catalog in order to agentically grant access to data. So this will be the only marketing slide that I show today. So this is the uh uh the platform. So we're really focused on provisioning access to data. And so we're building out this core AI layer that is today going to be focused on how can we assist the humans today to make faster decisions when humans are requesting access to data.
How can we assist humans on writing and authoring policy which may be complex? Uh how can we have an AI agent automatically force a reertification process to make recommendations of removing access to data for people? And so this is essentially the uh the platform today. And actually what we're very excited to announce uh last week was our first really big foray into this when I talked earlier about this idea of classifying access requests at kind of low on a scale up to high access requests.
This requires a lot of information. This requires information about the users, who they are. It requires information about the data. How has the data been classified? But also what we're looking at is what have your historical decisions been. So as a human, have you typically given access to a person that looks like this to data that looks like that? And what were the reasons you gave access to that? Or did you deny access because of a certain reason?
And so we're actually taking all that information. We want to look at historical reasons you gave that and we're going to build that into our model to make a recommendation. So even from the get-go, even if you're not looking at agents accessing data today, if you're looking at humans, we want to use AI to help again scale the approver. So the approver can come in and say, "All right, I already trust this and I want to automatically grant access to all these low-risk access requests.
Anything that's high-risisk, I want to forward on to the data steward or someone in the in the data domain because they can make a more intelligent decision. But the same system is also going to be in place when we start talking about agentic access to data. And so you might have you built an agent and using the MCP protocol engaging the uni catalog to say I need access. I found this data. It seems fit for my purpose for my use case and so I want to get access to it.
That same MPC agent's going to be told hey you need to go over to request access again via MCP which is an endpoint we announced last week. uh it's going to make that request and then you can tell immutant or the immuta agent and you automatically you are given authority and autonomy to automatically grant access to low-risk requests on that scale and so it'll take that action on your behalf or forward it to Slack say hey this is a medium or high request to the data steward you know we recommend you take this kind of action here here are the details about the interaction tell me what to do or go do it yourself and then you'll be granted access that agent will be granted access immed immediately.
So there isn't a ticket based system to this. It's agents automatically interacting uh within each other and to each other to automatically provision access again to develop and uh automate its task. And so if you actually want to see this in action, so we're at the booth uh E320 literally right over here behind the Fox Sports sign. We're demoing this off this week. So you can actually see a a Claude agent interacting just via via text. a little chatbot to make automated decisions based on classifications.
And so it's really this this idea of okay, how can we really start to see what this is going to look like in the future for not just our humans but our agents. So that's what I want to talk I'm happy to step offside if anybody has any questions about this idea of data provisioning either automated to humans or automated through agents. Uh but again I want to thank you for your time. If you wouldn't mind filling out the survey uh at the end of the session and be happy to answer your questions uh at the back of the room.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.