Building Secure AI-Powered Analytics Experiences Across Organizational Boundaries

Summary

Databricks staff engineer DJ Sharkey introduces Genie Agent Sharing, a feature that lets data providers share a Genie AI agent—not just tables—with multiple recipient organizations simultaneously, so business users on both sides can query combined data through a single natural language interface.
The feature extends Delta Sharing's zero-copy, cloud-agnostic data exchange protocol to include the provider's AI context, business rules, and governance controls, which travel with the shared agent to every recipient organization.
A live demo in this video shows a provider-side Genie agent being configured and a cross-side query that combines provider and recipient data, illustrating a shift from selling data as a product to delivering AI-powered data products as a business model.

Building Secure AI-Powered Analytics Experiences Across Organizational Boundaries

Data sharing between organizations has come a long way — but the next wave is about much more than moving files or granting table access. It's about delivering rich, intelligent experiences to partners, customers, and collaborators while keeping sensitive information protected on both sides. In this talk, we explore how the collaboration landscape is evolving: smarter defaults, stronger privacy guarantees, and AI-powered capabilities that make shared data more useful without increasing risk. You'll hear about the engineering principles behind building privacy-safe collaboration features at scale — balancing openness with control, and convenience with governance. Whether you work with external partners, internal business units, or cross-industry data networks, this video offers a look at where modern data collaboration is headed. Plus, see an early look at what's coming in a live demonstration.
Talk By: DJ Sharkey, Staff Software Engineer, Databricks ;

Chapters

FAQs

What is Genie Agent Sharing and how is it different from Delta Sharing?

Delta Sharing is a zero-copy protocol for sharing data tables across organizational boundaries. Genie Agent Sharing extends this by allowing providers to share a full Genie AI agent—including the provider's context, business rules, and query capabilities—so recipients can query shared data through a natural language interface without needing to understand the underlying schema.

What problem does Genie Agent Sharing solve for data recipients?

Data recipients often have access to shared data but their business users cannot actually use it because querying it requires technical knowledge of the schema and data structure. Genie Agent Sharing provides a natural language interface that allows business users on the recipient side to ask questions and get answers without any data engineering involvement.

How can one Genie agent be shared with multiple organizations at once?

Genie Agent Sharing allows a single provider-side Genie agent to be shared to multiple recipient organizations simultaneously through the Delta Sharing Marketplace. Each recipient organization gets access to the same AI-powered experience, with the provider's governance controls and context enforced for every query.

How does Genie Agent Sharing change the data provider business model?

This video describes a shift from selling data as a product—tables and files—to selling AI-powered data products through governed Genie agents that deliver insights. By sharing the intelligence on top of the data rather than the data itself, providers can deliver more value to customers while retaining control over how their data is interpreted and queried.

Full transcript

[00:08] All right, I think I'm live. Yes, people can hear me. Cool. Yeah, I'm gonna I guess I'll go. All right. Hi, everybody. I'm DJ Shy. I am a
[00:20] software engineer and the tech lead for Delta Sharing Marketplace and Clean Rooms. Quick fun fact about me. I deeply dislike public speaking. So, this is my idea of a very great time because I'm
[00:33] here with all of you. So, thank you for attending. Excited but nervous. Um, all right. Now that that's out of the way, today I want to talk to you about building secure AI powered analytics
[00:45] experiences across organizational boundaries. That's a mouthful. That's a really long title. Uh, you might be asking yourself, what what are you really talking about? What's today's talk about?
[00:57] So
[01:09] business users. Uh what does that mean? Pretend you're a 3P data provider and you want to share a Genie agent with your customers to help them unlock, you know, decrease time to value with the data you've shared. or
[01:21] alternatively you're a 3P data recipient and you really wish your business users could actually use the data which you've shared with them or which which they've purchased from you. So to explain what this is and why you should care, I'm
[01:34] going to bring you on a little journey about how this actually works today before our product. All right, as with all wonderful, you know, journeys, it starts with data. AI has a data problem.
[01:49] You've probably if you hang out in the same like parts of LinkedIn and the internet I hang out in and given the fact you're all at data and AI summit I'm sure you do you've heard the this saying before AI has a data problem. Uh
[02:02] what does that actually mean? It's AI and you know agents in turn are only as smart as the data they can access. So whenever someone says, "Hey, how can I make my agent better?" What
[02:16] they're really asking themselves is, "How can I get my agents access to better data?" And a common answer to that, the, you know, as the tech lead for Delta Sharing, one of my favorite answers to that is delta sharing,
[02:29] solving AI's data problem. So for those of you who don't know what Delta sharing is really quickly, it's a cloud agnostic no zero open source protocol for sharing data across
[02:41] companies and organizational boundaries. Uh delta sharing is great. It's an awesome way for to provide or receive 3P data and in my opinion it is a key part
[02:53] of solving AI's data problem. You need more data, you find a 3feed data provider who who has what you want and you integrate it and you get it. There you go.
[03:05] Cool. So, I need a data. What do I do with it now? I'm a business user. I just received some 3P data. What do I actually do with it? Well,
[03:17] in the old kind of pre-ai, pre- aent world, you would go to your data engineer and you'd say, "Hey, can I could you help me out? I I I need, you know, I need some insights from this 3P data." And your data engineer would say,
[03:32] "It's going to take time. This is way harder than you think it is. Make a ticket. Put it in my backlog." I'm the data engineer, by the way. I'm the guy who who says that all the time. People love me. uh we don't like that world. We're
[03:45] trying to move past that world. We're trying to let you know business users get direct access without actually going through this painful process. So, how do we do this? Simple solution. It's 2026. AI is awesome. Query it with an agent.
[03:59] Query your 3P data with an agent. Uh so, let's see what that looks like. So here we have an example where maybe you've reached out maybe you're an advertiser you've reached out to a
[04:11] publisher and you got some some information about how your campaigns are performing. So you come in and you say, "Hey agent, which of my I got all this great 3P publisher data. Which of my campaigns is performing best
[04:25] and the agent comes back very convincingly?" And says, "Oh, well, I joined the clicks table to the impressions table and I found, you know, the average session session duration for
[04:37] summer sales 2024 was the highest. Therefore, that's your best campaign." This is awesome. We data engineers out of the loop, business users getting direct access to insights, you know, self-service access to their insights.
[04:51] But there's a catch. It's not as awesome as it seems. So, what's that catch? Well, agents lack business context. What is performing best even mean?
[05:03] Before we said it was session duration, but is a longer session duration good or bad? Uh, you know, maybe it means there's really high engagement or maybe it means your page load speed is really
[05:15] slow or just your customers are really confused and getting stuck on your landing page. Uh, do all campaigns even have the same goal? Could you even use the same metric for measure for comparing these
[05:27] different campaign types? Um, not clear. There's more confusion. Agents don't even have data context. What do I mean by that? Earlier your agent said, "Hey, I joined on landing page URL." Cool. Is that the right join
[05:42] key? Is it a unique identifier? Is it a onetoone relationship? Should you use a left join or a right join or an outer join? These are questions that your business user doesn't want to answer. So your business user is, you know, they're
[05:55] tired. They don't they don't care about this. So before we had these awesome results, they looked so good. But we start looking at this and we start noticing maybe we can't trust these results. Maybe they're not all they're
[06:07] worked out, not all we hoped they would be. And now we're sad. Now we're sad because we don't trust our agents response. So before we said AI has a data problem,
[06:20] but really my new quote, please, you know, everyone put this on their LinkedIn, credit me. Everyone's going to be saying it soon. AI with data has a context problem. Um,
[06:33] so how do we solve that context problem? How do we help, you know, given AI now has access to all your data, all your three great 3P data, how do we actually help our 3P data recipient users utilize
[06:46] it with the proper context to come up with meaningful results.
[06:58] everyone caught it, but uh I think last week we officially announced a rebrand. Uh Delta sharing is now called open sharing. Uh this is kind of a double down on our commitment to both open source and to moving past sharing just
[07:11] data to also sharing all sorts of things including any AI asset um and sharing context which helps AI utilize that data. So
[07:24] from the people who brought you Genie agents and open sharing, we proudly present Genie agent sharing as I promised that would be the topic of today's talk. So what is Genie agent sharing?
[07:38] Genie agent sharing helps bridge the J gap from 3P data to your the actual recipients business users and it's all based on sharing context. Uh with shared
[07:51] context, your recipients genie agents automatically take advantage of all that domain expertise which you as a 3P data provider have. They don't need to recreate it. After all, who knows the
[08:03] data being shared out better than the data owner, the 3P data provider. Um so what is it really? Well, as a data provider, you share a curated natural language agent built on top of provider
[08:15] data and domain context. on the screenshot on the right, you know, you see you put in a whole bunch of general instructions that are saying, well, this is what a genie or this is how the genie agent works. This is, you know, what you need to know about my data. All that's
[08:28] going to end up being shared to your recipient. Uh, so as a provider, oneclick share data and its context. You get centralized management of what you're sharing. So, it's easy to control and easy to share across many
[08:40] recipients. And soon, you'll get data controls, which I'm going to focus on more at the end here. >>
[08:54] user ready agents. They get one-click agent creation preloaded with that provider data and all that great provider context. They get a interface that they already
[09:06] know. This is identical to the Genie agent interface for that for one P agents now preloaded with all that 3P g greatness and they get the ability to bring their own data and context into it without
[09:20] exposing any of that back to the provider. That's really powerful. I'm going to touch on that again during the demo as well.
[09:32] kind of an architecture diagram that we put together for it. Key things I want to call out here. As your end user is interacting with your Genie agent, your Genie agent is in turn interacting with provider data provider context and
[09:46] recipient data, recipient context. Or if you have a more technical user, you could kind of use the traditional Delta sharing way of just getting direct SQL access to that provider data. So in a nutshell, that's what Genie agent
[09:59] sharing is. So I'm going to move over here. We're going to do a quick demo, live demo that, you know, I'm sure is going to go perfectly. No one's ever had a problem with a live demo before. All right, give me a second here.
[10:18] Cool. Okay, so here what I have is a provider Genie agent. And what I mean by that is I'm a 3P data provider. I have manually
[10:30] curated this genie, this genie agent. It has access to a whole bunch of my tables. It has access to a whole bunch of text instructions explaining, you know, how to use those tables. It has information
[10:44] about, let me expand this, information about how these tables can join together, common SQL expressions and what they actually mean, like how to calculate total revenue based on these tables. and even some example SQL
[10:56] queries. Just a whole bunch of context about what do my tables actually mean? So when my recipients go to use them, they could use them easily and get, you know, good answers on their first shot.
[11:08] So
[11:21] to three different companies all in one go. Like I said, centralized management Click the share button. And now I'm going to switch tabs. Apologies. Switching tabs. I am in my a
[11:34] different databicks account now where I could see, hey, look, the provider has shared a Genie agent with me. This is cool.
[11:46] Click this mount button
[11:59] And in a second, if all goes well, this will take me directly to the Genie agent page. There it goes. Thank goodness. All right, cool. So, here I'm in my recipient account. I just purchased the
[12:12] 3P uh a data agent from my provider. And I click configure and I could actually see it's preloaded with all of that data. That's cool. Preload with all
[12:25] provider data. It's pre-loaded with all the providers context. Also really cool. Um let's see here. Pre-loaded with our SQL expressions, their SQL queries. All great stuff.
[12:37] Just a moment. All right. So now what I'm going to do, and I'm sharing the wrong screen, are I? Cool. Don't worry about that. Now what
[12:49] I'm going to do is as a recipient, I'm actually going to add my own data to this agent. So I'm going to go here. I'm going to go to my advertiser catalog. I'm going to add all these tables in.
[13:20] campaign had the best purchase rate postclick? Oh, forgot to click the save button. My bad. And that probably made meant I'm going
[13:33] to reask that question. Apologies. All right. Which ad campaign had the best purchase rate postclick? So, what makes this an interesting question? Well, the publisher is the source of
[13:45] truth for, you know, how my campaign like who's seeing my campaign, how who's actually clicking those, but I'm the source of truth as the advertiser. I'm the source of truth for who's actually transacting, who's actually buying something afterwards. So, I think that's
[13:59] a really interesting question because it requires an agent having access to both recipient side data and provider side data. recipient side context and provider side context.
[14:11] There we go. So, it came back. It figured out, hey, spring sale 2026 did awesome. That's your best campaign. You know, if you're measuring by purchase rate, we can even click into the source and we can see it's actually joining
[14:25] from the provider tables to the advertiser tables. It is taking advantage of both my data set and my provider's data set. This is really cool. This is a marketing user who doesn't have any technical expertise
[14:38] getting immediate immediate value from 3P provider data without needing to do anything. They didn't have to write this complicated SQL query. Honestly, I'm glad I didn't have to write this complicated SQL query.
[14:56] So, that's the quick demo. Give me a second. I'm going to reboot my slides here. Uh you might be asking yourself when is this available? Does this actually exist? Yes, we launched started
[15:08] launching our beta yesterday. Uh couple day roll out, you know, be safe. So it'll be fully available everywhere tomorrow, which is we're really excited about. uh and even have a launch
[15:22] partner, Cathy Labs, uh really kind of driving home the idea that it's not just about providing data anymore. It's about shortening the distance between a question. Oh. Oh, no.
[15:34] It's about shortening the distance between a question and a decision. It's about decreasing that time to value. And just one quick second. I'm so sorry. I've lost my speaker notes because I clicked the wrong button.
[15:59] All right, fantastic. There we go. Cool. So, now that we've seen that, uh, I want to switch gears a little bit to talk about those data controls which I mentioned earlier. So, GI agent sharing data controls. Um, it's an upcoming feature that lets providers put limits
[16:11] on how recipients can use can use the agent and use their data. You can do things like make your data accessible only via the agent. Put limits on the number of conversations your business the end recipient business
[16:24] users can actually ask or put limitations on the number of bytes a response returns the number of rows a response returns. So really what we're talking about is you
[16:36] know back to this architecture diagram you're only allowing access via that agent and it's kind of bottlenecked access. you could put restrictions on what exactly that means. No more direct SQL access. This is a huge departure
[16:48] from a traditional data sales kind of business model. Uh and I think this is an opportunity for us to reinvent traditional data licensing. I think it's super powerful. Uh maybe you've all been
[17:00] involved in selling data before. there's this really kind of challenging sales cycle, a push and pull between a provider and the recipient where the recipient understandably wants some level of confidence that the provider's
[17:13] data is going to help them that it's going to answer their questions ahead of time. Hey, do you cover the segment I'm interested? Do you have interested in? Do you have data about my actual users, about my actual region, whatever it is? Uh, and the data providers
[17:25] understandably don't want to give access ahead of time, right? They want to sign a contract before they give access to their data. So simple questions like does this data set cover my portfolio can end up being really hard to answer.
[17:38] Um are my target customers in your audience? These are really hard questions to answer. Without access to the entire data set and even if that access is granted, you end up writing a lot of SQL. You end up doing a lot of data engineering work.
[17:50] It's hard to map the schemes together. It's it's hard. You get a really long sales cycle as a result. >>
[18:02] lay the groundwork for a brand new business model where you could actually pay per question or use this as like a pre-sales thing where maybe you're not paying at all initially but you just have access to this limited set of
[18:14] business user questions necessary for your pre-sales journey. Um, so yeah, it's a really exciting time to be a data provider, really exciting time to be a data recipient and in general, I
[18:27] think Genie agent sharing is a great way to unlock or to make much faster time to insights for end business users and eventually unlock new business models as well. All right. Well, thank you
[18:40] everybody.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.