Solving Exclusive Data Access With Role-Based Access Control

Summary

• Exclusive role-based access: Users wearing multiple hats can switch roles in the Databricks workspace to work exclusively on dedicated projects or client data.
• Healthcare compliance: This capability solves critical problems in healthcare where clinical trial data cannot be combined and must be isolated by project.
• Data governance: When switching to a particular role, the workspace adapts so only objects and UC data of that specific role are accessible, enforcing exclusive access.

Solving Exclusive Data Access With Role-Based Access Control

Talk by Siddharth Bhai, Product Management, Databricks ; Stefania Leone, Director, Product Management, Databricks
Do you have users that wear multiple hats over a day? Like working with data from various customers and hoping they don’t inadvertently aggregate data? Or are they working on sensitive datasets such as clinical trials that should not be combined, or are data sets that are subject to regulations? We have a solution! In this video, we will present a new capability that allows users wearing multiple hats to switch roles in the Databricks workspace to work exclusively on a dedicated project, data of a particular client or clinical trial. When switching to a particular role, the workspace adapts in such a way that only workspace objects and UC data of that particular role are accessible. We will also showcase the administrative experience of setting up exclusive access using groups and UC permissions.
Here’s more to explore:
Unified and open governance for data and AI: https://www.databricks.com/product/unity-catalog
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

Chapters

• 0:00 Introduction and challenges
• 2:33 Evolution of permissions
• 4:40 Role-based access control
• 7:15 Demo: Workspace access
• 11:03 Demo: BI tool integration
• 12:26 Auditing and monitoring
• 13:39 Implementation steps

FAQs

What problem does exclusive role-based access solve?

Users in healthcare, consulting, and regulated industries often work with multiple projects or clients. This video shows how role-based access control with Unity Catalog lets these users switch roles to work exclusively on one project's data without risk of inadvertently combining sensitive datasets.

How does this work in healthcare?

In clinical trials, data scientists often work on multiple trials but must keep trial data isolated to avoid bias and regulatory violations. Role-based access control lets them switch to a trial-specific role that only shows that trial's data, ensuring compliance and data segregation.

How do I implement attribute-based access control on data?

Role-based and attribute-based access control on Databricks Data and AI platform enforces data access policies at the table and row level. Organizations define rules based on user attributes, data classifications, and business logic—then Unity Catalog automatically enforces these permissions, ensuring users can only access data they're authorized to see.

best platforms for data access control

Databricks Data and AI platform with Unity Catalog provides comprehensive data access control through role-based and attribute-based policies. The system enables fine-grained permissions at table, column, and row levels, ensuring secure data sharing while maintaining operational efficiency and compliance.

What other industries benefit from exclusive data access?

Consulting firms need to isolate client data to prevent conflicts of interest, banking industries need to separate data by country, and regulated industries need to keep restricted and unrestricted datasets separate. Exclusive role-based access enables all these use cases.

Full transcript

0:00 Introduction and challenges

Welcome everyone to this session. Um today we're uh we'll be talking about how we solve exclusive access with role-based access control. Um. I'm a product manager here at datab bricks working on unity catalog and the data bricks runtime. And with me is. He's also a product manager at datab bricks working on IM and U. Um so maybe before we start like who of you um has used data bricks before you see okay about half the room yeah half the room okay um thanks uh okay so what is exclusive access um so we talked to many customers and there's uh examples across many different industries that have very similar characteristics for example in the healthcare industries a lot of our customers work with uh clinical trials.
So data scientists in this case like uh is working on different trials. She's working on trial one on trial two. She has access to some other data um but she shouldn't be like kind of mix and matching like trial data for like various reasons. Also in the consulting industry um some of our customers um have solutions where they work with data from multiple clients. So, a consultant may be working on data of Coca-Cola, but then they're also consulting Pepsi.
And of course, they shouldn't be able to mix those two data sets while still being able to access it. Then um in like regulated industries sometimes there's like restricted um data sets, unrestricted data sets and while users that are working with unrestricted data sets in some circumstances they should actually be able to also access restricted data but without mix u matching um doing toxic joins across those two. or in the banking industry there may be for example account data from like bank customers for like various countries and you shouldn't be able to join them.
So those were the kind of like feedback we got for like this request for like isolating different data sets from each other and with that also the permission. So it kind of boiled down to um based on the example of the healthcare industry like should be able to work on multiple trials but only one at a time. So how was that solved pre-Unity catalog? So for those of you that were around before we had Unity catalog, we had what we would call clusterbased permission.

2:33 Evolution of permissions

So how would access data? This was configured through a cluster. So you could have a cluster with an instance profile that wraps an IM role or a service principle and that role was configured to access a certain um part or like certain data. Um and then for another cluster you could configure an instance profile to access other data in this example trial two and alice would gain indirect data indirect access to that data through can attach on cluster.
So she would attach the notebook to cluster one and get access to trial one. She would attach the notebook to cluster two and get access to trial two. So you had a very clean isolation. Um but the access was indirect. Uh so when we introduced Unity catalog, we changed that permission model to a userbased permission model where permissions follow the user. So if I have been granted access to something, I can access the data independently of the compute I'm running on.
Same for. So if we would model um the example from before like would be we would have like groups for trial one, trial two, group one and via membership would actually get access to all the data those groups have been granted access to. So with unity catalog like in this example, would have like the union of all the permissions of like the permission she has been granted um like personally or permissions that have been granted to trial one or trial two or group one.
So this is uh this requirement doesn't go well with like the requirement uh or this setup doesn't go well with the use case of exclusive access where we actually don't want to have like the union permission over like all these trials. We only want her to have access to one at a time. So how are we solving that? So we're access uh we're solving that with um role-based access control. So role-based access control is kind of a big word.

4:40 Role-based access control

Uh if you ask any AI out there, you'll get like a definition like that. So role-based access control is like a model where to manage permissions. So each role represents a set of permissions kind of a container and then user gain access um to those like resources by being assigned to the role. So what is interesting here is like the second part uh in some systems users can assume roles temporarily um to gain access to some data and they they can um lose those permissions again and this is exactly what we're going what we have used to um uh realize exclusive access.
So what we're introducing is um exclusive groups. So we today we have like the regular groups with this union semantics uh of permission where if you're a member you basically um get all the permissions of all the groups you're a member of uh with these new exclusive groups instead of adding permission to the user the group replaces a user's permission. So what does that mean? So if is a member of group one and can assume trial one which is an exclusive group, she either can act as and have like the union of permission of all her permissions and the group permission or she can act as trial one and only has trial one permission.
So trial one replaces the permissions she's been granted. What would that mean? So how would we model the scenario we had before? So trial one and trial two would be modeled as exclusive groups. Um and then may be member of like some other groups like group one here. So Alice can either um authorize as herself then she has her own permissions and all the permissions she's been granted access to in this case group one or she assumes trial one by which she authorizes as trial one group and then she only has trial one group permissions.
Then she can assume trial two. Then she only gets trial two permission. And with that we get like a very clean isolation of data access. This is a lot of theory. Now Sid will show you how the product experience will actually look like. Thank you so much. Yeah, it's exciting to see all of this attention after lunch. There's claps already. That's great. Hopefully those will continue after the demo. I see a few folks standing in the back.

7:15 Demo: Workspace access

Don't be shy. We have some empty seats in the first three rows. If you all want to come up while I get the demo set up, please feel free. So, walked you through. It's a very new paradigm we're excited to introduce to a set of features we have in Unity catalog for fine brain access control. And we can definitely stay back and chat about some of the other ones after the session. But since we're short on time, I'm going to jump straight into a specific use case.
So, the demo that we look at is going to go through four steps. We'll login as ourselves into the workspace and see what you can access. Then we'll go ahead and see the user switch out to working on trial one just like the example that shared. Then we'll switch out as working on trial 2. And then we will see how can you get insights powered by data bricks even when you use other third party BI tools such as Tableau. So let's go ahead and jump in.
So you see me starting off in a UI that will hopefully feel familiar to everyone in the room. I'm logged into the databicks workspace as myself and as I go and start my day I get to my home folder and I have pre-created a query as I go into it you can see it's a very simple query select star from a table in the trial one schema pretty straightforward now what do you think should happen when I go ahead and run it should actually fail and why is that because I'm acting as myself so if I scroll down and I expand the permissions, you will see it correctly complains the user does not have permissions on the schema.
Great. So that's what we wanted because you shouldn't be able to get to it as yourself. So let's go ahead and take a look at how the data is organized. You can see I have a trials catalog and then I have a schema and then tables for each month's experiment results. I as the user can look at the metadata because I have the browse privilege in Unity catalog, but I can't get to the data as myself. So, let's go ahead and reset.
I'm now back into the workspace, but I'm going to go ahead and switch out to working on trial one. And this is a very important foundational step because I've now gone ahead and switched out my permissions to authorizing as this group trial one. This time when I come into the workspace, you can see there's a different folder for me. And this folder maybe has some information related to the trial such as data I may be working with or images.
Maybe I have a couple of notebooks that I might be using for my analysis. And then I have the same query that we just tried to run as the user, but this time it succeeds as we expect because we have now switched our authorization context to trial one. So, it works as expected. And let's say a couple of hours go by. I do some more analyses on trial one. And now it's time for me to go ahead and work on a different clinical trial.
So, at this point, I'm in the same workspace. I am the same user, but I will go ahead and switch out my context from trial one to trial 2. So, behind the scenes, my authorization context in data has switched to working on trial 2. It's a pretty similar folder structure that you see again a couple of notebooks, a query, but there's a difference. The query this time when I go ahead and open it has me working on a similar name table but for a different schema trial 2.
And as I go ahead and run this, it works successfully. But because I'm acting as trial one, sorry, as trial two, if I was to try and query trial one data, it would not succeed. So this is how you're able to go ahead and segregate access. But a lot of you are excited to work with data bricks from other tools. And here you can see me going to Tableau and going through our recommended ooth powered databicks signin flow.

11:03 Demo: BI tool integration

Now when I go ahead and do this, it brings up a web browser. I go ahead and authenticate as myself and after that I get presented an option to continue on either as myself as the user or to go ahead and switch out to trial one or two. So let's pick it up from trial 2, which is the exact same experience we were in from inside the workspace. So now when I go ahead and come into Tableau, you can see I can get to the same trials catalog which we looked at from catalog explorer a couple of minutes ago.
And now I can go ahead and query for trial 2 schema. And when that comes through from inside Tableau without needing to go to the databicks first party UI, I can search for tables that start with trial 2. So when I go ahead and run, you see I have similar named tables and I can now once again get to the same main results table that we were querying through the databicks UI. So this is how you can see this mechanism goes ahead and lets you have access from either the databicks UI or third party UIs of interest.
Did that seem interesting? Okay. But I know a lot of you are security focused. So I want to make sure we cover the auditing piece. Well, now there's something new in this audit event. Can someone spot it? There is a run as and there is a run by. So you can see the run by reflects my username because I was conducting the queries as a user. But look at the run as there is a trial two and a trial one. So this cementss the notion that you're authorizing to unity catalog with that specific trial.

12:26 Auditing and monitoring

And now you could conduct all kinds of analyses such as what are all the things this user accessed or when access authorization happened which specific context was used. Now how do we go ahead and set it all up? It's very simple. Three steps. First we go ahead and define these exclusive groups. Earlier in the presentation, you saw Stefani explain how we have regular groups differentiated from exclusive groups. I have a visual on that coming up in a second.
Step two, you go ahead and give access to your data to these groups. And step three, you take your users and identity provider groups and give them the ability to assume these new exclusive groups. Let's take a look at each one step by step. Does this UI seem familiar to folks in the room? Yeah, a lot of heads nodding. Great. This is inside the account console and I'm working with the groups in my account. But you'll notice at the bottom of the table, I have two new groups, trial one and trial two.

13:39 Implementation steps

And I'm I have those designated for exclusive access. Rest of the tooling is going to feel very familiar to you. The next slide is hopefully going to be even more familiar to you because you are now giving permissions inside UC and you see me with three ground statements. The first one grants use catalog on trials to trial one, the exclusive group we defined in the previous step. The second one gives me the ability to now use the schema that corresponds to trial one.
And you would have a similar grant statement for trial 2 if you were writing up those permissions. And the third one gives me the ability to query the table that we queried earlier in the demo. The interesting thing is this entire tooling is exactly the same as you experience with Unity catalog today. And if you have existing scripts that you're using to manage UC, those will work with no changes because it's just a group for us behind the scenes.
The third step is pretty interesting to try and get this working in your enterprise environments. So at the bottom half, you can see perhaps an identity provider configuration that it feels familiar to. You have users, users are inside IDP groups and you've gone ahead and onboarded them to data bricks. You can use scheming or you can go ahead and use automatic identity management if you're an Azure. The part at the top also we just covered in the previous slide which is these are the schemas and then these are the exclusive groups that may query them.
The middle part is the new piece where you have this ability to grant can assume to users or groups. Now what's interesting is that the groups that you're giving can assume to can be synchronized from your identity provider. This allows many of you who may be using workflows within your IDPS to add and remove users from IDP groups to be reflected directly in data bricks. So those are the three steps as you bring it all together.
You can define exclusive groups. You can grant access in UC to those exclusive groups and then you can give permissions to users or IDP groups to assume those exclusive groups. For those of you who have cameras out, yeah, please go ahead and scan that QR code. It'll take you straight to our preview form where you can go ahead and let us know if you'd like to be on this preview. And with that, Stefani and I will open it up for questions.

Learn more about the Databricks Data and AI platform.