Building Intelligent Data Discovery with Unity Catalog: Domains and Discover

Summary

This video covers how the Discover page, Domains, and Glossary in Unity Catalog work together to help data practitioners, business users, and AI agents find relevant data and AI assets across an enterprise account without getting lost in an ungoverned catalog.
Domains provide a business-aligned governance layer that groups data assets into meaningful organizational units with subdomains, while the Discover page curates certified, trending, and personalized assets with a customizable layout for specific events or promotions.
A Glossary feature provides authoritative business concept definitions that serve as a curated context layer for AI built on the Databricks Data and AI platform, improving the accuracy and reliability of AI-generated answers.

Building Intelligent Data Discovery with Unity Catalog: Domains and Discover

As data estates scale across teams and workspaces, governance and discovery must operate at the account level, not just within individual projects. In this video, we’ll explore how Discover Page, Domains, and recommendations in Unity Catalog work together to scale data governance and discovery across the enterprise. You’ll learn how Domains provide a business-aligned governance layer, how Discover Page curates trusted data and insights across the account, and how recommendations leverage metadata, lineage, and usage signals to surface the right data at the right time. We’ll share real customer patterns and practical guidance for building a governed, intelligent discovery experience that grows with your organization.
Talk By: Qihua Wang, Sr Eng Manager, Databricks ; Qihua Wang, Sr Eng Manager, Databricks ;

Chapters

FAQs

What is the Discover page in Unity Catalog and what problem does it solve?

The Discover page is Unity Catalog's one-stop browsing experience that surfaces certified, trending, and personalized data and AI assets across the entire account. It addresses the challenge of both humans and AI agents getting lost in large catalogs by curating the most relevant and trusted assets rather than presenting a raw, ungoverned list.

What are Domains in Unity Catalog and how do they support governance?

Domains are business-aligned groupings in Unity Catalog that organize data assets into logical units reflecting the structure of the organization, such as by function, product line, or business unit. They support governance by providing a layer above individual tables and schemas where policies, ownership, and discovery can be managed at the business level rather than only at the technical level.

How does the Glossary feature in Unity Catalog help AI applications on Databricks?

The Glossary provides authoritative definitions for business concepts that serve as a curated context layer for any AI built on the Databricks Data and AI platform. By giving AI systems access to agreed-upon definitions of key terms, the Glossary improves the accuracy and reliability of AI-generated answers that depend on understanding what specific business metrics or entities mean.

What is on the roadmap for Unity Catalog data discovery features?

This video mentions Genie-powered discovery as a roadmap item, where conversational AI will help users find the right data assets through natural language rather than browsing alone. Data products are also referenced as a future direction for creating AI-consumable, governed bundles of assets within the discovery experience.

Full transcript

[00:08] Hello folks. Good afternoon. Thanks for joining us today. Uh I'm Steph. I'm a product manager here at Data Bricks working on our discovery domains and uh some of our glossery and ontology initiatives. And uh with me we have Shiua who's our awesome engineering
[00:20] manager who all who brings all of this to life. Um so before we get started just our forward-looking statement. um take a quick peek, but TLDDR just uh this isformational only. Um none of this
[00:32] is promised. These are things that we're going to be sh we're going to be sharing things that we're considering, some of them that we're actually building. Um so just just a heads up. Um and then please complete your surveys when we're done.
[00:44] Um feedback is really really helpful. Helps us understand what sessions are valuable, how we can make them better. So please, please do take a few minutes at the end and uh and complete the survey. So with that, we'll kick it off. So,
[00:56] Shiu and I are going to be talking a little bit about how you can build a uh an intelligent data discovery experience with Unity catalog with a focus on our new domains feature and our new discover uh feature. Um we'll do we have a couple
[01:10] of slides to start and then um Chiua has prepared a lot of awesome demos which uh hopefully we'll I'm sure will bring everything to life and uh then we'll wrap up by sharing a little bit more about what's on the horizon some of the
[01:22] things that we're continuing to invest in and uh then if we have time we'll open it up for Q&A but we're also happy to take questions uh after the session or offline we'll leave our contact info up on in a QR code at the end.
[01:37] Cool. So maybe just to ground like what we hope you all walk away with today. Um two key things that we want to show you. The first one is how can you build a better discovery experience for the members of your organization. The key
[01:49] goal here being allowing your data practitioners, business users, and your agents uh to find data and AI AI assets that are actually relevant to them. And then, as you probably heard from Ali
[02:02] this morning, one of the things that we're really trying to do is help you build build a a curated context layer in addition to our automated context layer. Um, so that any AI that you build on data bricks is accurate and reliable.
[02:16] Those are the two things we want you to walk away with today and we'll show you how to do that. So maybe we can take a step back and just like what is this problem? What what is the problem that we're solving for in the first place? But as I'm sure you all are aware, um especially in this
[02:30] AI era that we're in, um navigating your data and AI estate um becomes increasingly difficult as your data estate grows. And up until recently, that was really only a human problem. But in this new agentic era, it's
[02:43] quickly becoming an agent problem, too. So maybe just to illustrate this with a light-hearted example, let's pretend you run a highly successful school of magic and you have a question, right? You want to understand like how enrollment by
[02:55] house has changed year-over-year. What we've heard from a lot of customers like yourselves is that if a human is doing this, it often turns into a goose chase, right? It might mean you go on like a detective work. You might go on a
[03:07] detective hunt, searching through Slack, digging through old threads, trying to hope hoping that someone has already answered that question, pointing you to the right table, the right dashboard, the right notebook. It might mean digging through some wikis who may that may or may not be accurate or
[03:20] maintained. It might mean doing some enterprise search across different tools in the hopes that you find something that is relevant or that is answering your question. Um, and what we also hear a
[03:33] lot is it means clicking through a pretty dense catalog, right? Like Unity catalog is great at helping you organize your assets, but it doesn't always help you translate and actually organize things according to use case or according to how your business is
[03:45] structured. And my personal favorite, you know, if all else fails, you just ping someone or you ping a channel. Like this is what happens most of the time. This is what I do and I feel really bad when I do it. Um you just ping someone, ask them, hey,
[03:57] what table should I use? Do we have a dashboard that answers this question? Um and you know, by the time that person gets back to you, it could be a day, could be a two days, but the point being that this is a highly inefficient uh
[04:09] process. And one thing that we've learned is that for agents, it's just as hard, right, if not harder. Um, and we've learned that over the last couple of years, and last couple of months, it's gotten even
[04:21] worse, right? Um, not only does first of all, an agent has to know what the right asset is to use. Um, same problem that a human has, right? But you might empathize with this, like you have 15 different tables that have some like
[04:33] variation of the same name. All of them could be potential candidates for whatever use case you're trying to answer. In this case, trying to find enrollment by class. But the even more challenging part and
[04:45] this is where AI is at a disadvantage compared to a human is that AI has to know what you also mean right so if I ask like I want to understand enrollment by house and how that's changed year-over-year my agent has to
[04:57] understand like what does enrollment mean right like do we count this at the beginning of the term the end of the term do we want to include transfers do we want to include withdrawals do we want to drop withdrawals what is a house is a house like a physical house that
[05:09] people are living in is it a student grouping Does Hagrid's house count? Um, and year, right? This is something that I think is more analogous to the world that we live in. But how do you define a year? Academic year. When does that start? When does that end? Calendar year. Um,
[05:22] all of these are critical in order for AI to actually for your agent to actually give you an accurate response. And the the challenging piece is that you know these pieces of context that are critical to ensuring the agent gives you a reliable and accurate response,
[05:35] they live all over the place, right? wiks, docs, but most commonly what we hear is in people's heads. And then this makes it very difficult to create AI that actually is grounded in your business context and can produce reliable results at scale.
[05:50] So just to put this all together, we know that in order for you to have really good data and AI agents, you need both good business context and you need the right data. Um, so what we're going to do today is how we can get you into
[06:02] that Leo DiCaprio quadrant and show you how you can create a system in data bricks that allows both your humans and the humans in your organization as well as your agents uh
[06:14] to find the right thing with the right semantic context so you can have reliable AI as well. So the answer to that is what we've so far built and this is building off of what Ali mentioned and shared this
[06:27] morning at the keynote. Um there are four key ingredients that we're going to talk through today um that we want to share with you so you know how to go off and actually create a context layer that allows both for your humans to find the
[06:39] things that they need and for agents to be accurate and reliable. We're going to talk in in about each of these in a little bit more detail. Um, but just to ground us, we'll start with talking about discover, which is the one place in the UI where your humans can go and
[06:53] browse everything that they have access to. Then we'll talk a little bit about domains, which allow you to group assets within your data estate into logical groupings that actually map to how your business is structured. Then we'll talk
[07:06] about the glossery which is a way for you to actually define that context layer where you can give authoritative source of truth definitions for business concepts. And then we'll we won't talk too much about genieontology today. Uh
[07:20] but just so to make sure you all see how this comes together. Geneontology is at layer on top which automatically learns knowledge because we know that's crucial if you want context at scale. Humans can only o only go so far when it comes to
[07:33] actually creating that context. So we'll focus on these three um and we'll we'll dive a little bit deeper into each of them and then Chiua is going to take us through an exciting demo.
[07:45] All right. So what is discover? Discover is a new feature currently in beta that we've built into the data bricks workspace experience. And the purpose of the discovery experience is to serve as that internal marketplace where all of
[07:58] your practitioners and your business users can go and they can browse all of the assets that they have access to. No more digging through a complex hierarchy or through a folder structure or clicking through cataloges and schemas
[08:10] and trying to figure out what the right thing is or what's valuable for them. Discover is the place where you can find those curated highquality assets that make sense for your use case. One of the crucial elements, and we'll
[08:22] dive into this in a little bit more, is domains. So, this is the place where you can go and browse by domains. Um, and I'll go into more depth on that in a second. Um, but then you can also discover and browse things that are
[08:34] curated, right? Things that members of your of your workspace have certified. You can find things that are trending. You can find things that are popular. So even if no people haven't gone in, even if your admin hasn't gone in and done heavy curation work, uh it surfaces
[08:47] those those things that are likely to be of value based on how others are engaging with them. We've also built in rich personalized recommendations. So if you go on discover, you'll see a set of assets that we've determined are likely to be
[09:00] of interest to you based on the things that you've engaged with in the past, based on semantic similarity, based on lineage and those relationships between the assets. So uh identifying those those high-quality assets that are
[09:12] likely relevant to you and upleveling them. And then lastly, this is something that is not only valuable and not only available for the technical practitioners in the workspace, this is also something that we're soon introducing into Genie 1. So if you want
[09:25] your business users to be able to browse the dashboards, the genie spaces that you've created for them and not have them to dig through lists or dig through hierarchies, discover is what makes it a super business userfriendly experience and allows allows them to browse and
[09:38] find things that are relevant to them um a lot more efficiently. So moving on to domains. As I mentioned earlier, domains are really like the crucial piece of the discovery
[09:50] experience that allow you to browse uh by business area. So as a curator, you can define what a domain represents and it might it might map to a particular
[10:02] business unit, a particular business area, a particular use case. There's lots of flexibility as you'll see in the demo. But the purpose of this is to create these logical containers and then you could add anything you want to them.
[10:14] Right? Regardless of where an asset lives, if it's a workspace asset, if it's a UC asset, if they live in various different schemas, domains are a way for you to effectively virtually collect group all of these things and make them browsable based on how they actually uh
[10:28] materialize and how they actually matter to you and as a business. Domains aren't only valuable for humans, they're also valuable for agents. So you can imagine an agent if you ask an agent a question about finance an agent will
[10:40] look and try to understand try to see is there a finance related domain and if it does find a finance related domain that finance related domain hopefully has some finance trusted sources data sources that it can draw from. So this
[10:52] is one of those key elements that provides context to ground your agent in the high quality sources that it should be drawing from as it is doing work for you or as it is drawing insights from you.
[11:04] And last but not least, as I mentioned, P practitioners and business users can um use this to find the trusted high-quality assets that are relevant to them by navigating according to business area and not having to navigate a technical hierarchy.
[11:24] And lastly, we have the glossery. And as we mentioned earlier, the glossery is that curated human-defined context layer where you can go in and you can define the authoritative source of truth definition for these business concepts
[11:36] that are critical to make your AI accurate and grounded in your business's actual reality. So as a curator, you can go in, you can document context, um certify business concepts, um Chihuahua
[11:49] will demo it in a second. Um agents again can reference these source of truth definitions of business concepts. So you could imagine if I ask an agent like how much did what was a revenue for the fiscal year agent might not know
[12:02] like how our fiscal year is defined. But if there's an authoritative definition for fiscal year and glossery that grounds the agent in an actual source of truth and increases the likelihood that the response from the agent is actually accurate. And then there's also the side
[12:16] benefit, right, that practitioners and business users, if they're doing their own work, um they can refer to those authoritative analyses. So if they're building dashboards, if they're building notebooks, they they can know what the authoritative definition for these concepts is to make sure that whatever
[12:29] they're building, whatever in whatever they're building is is actually aligned with how the organization defines these concepts. >>
[12:41] together, this is you may have seen the slide already, but this is like what we've been calling our semantic context stack. And in the middle, you'll see that we have domains, glossery, and metrics as those human curated
[12:56] authoritative pieces of context that you can build into data bricks to ensure that you have a richer, more accurate and more reliable AI experience. And as I mentioned, Genie ontolo ontology is
[13:08] built on top of all of that as well, right? So, Genie extracts these knowledge snippets uh automatically learning knowledge to ensure that in addition to the context that you are manually curating within UC, um you can
[13:21] have context at scale cover all your bases and of course it's doing that only if it has extracted knowledge snippets with a high degree of confidence.
[13:33] So that's a quick overview of what discover domains and glossery are um and how they all make up our context stack at data bricks. Um next up we'll move into a live demo so you can actually see all of these things in action.
[13:46] >> All right, cool. Thanks Steph. >> Okay. >> Um yeah, so uh here we are at the uh the the home uh the the homepage of uh data
[13:58] bricks. Um and then like uh we can find the uh discover just on the n left. Uh for like folks who are already on the uh beta of discover you can already see like the options uh in your left nap of
[14:11] your workspace. Uh but uh we are launching public preview next week. So like uh it will actually be available to uh most workspaces by next week. Okay, let's go to the discover page and see
[14:23] how it looks like. Um so this is the discover page. uh at the top we have like search bars and uh the filter panels um I will showcase that uh later and then below that uh at the top we
[14:36] have uh a domain session that are going to allow users to uh browse the uh assets in organizations by the domains uh that are curated and uh I will uh actually uh show more
[14:49] details about the domain session uh later and then below the domain session we have uh a session on certified access. Um so like these are the uh high quality trustworthy access that are um certified
[15:04] by like the data stewarders in your organization. Um so like there are four highlighted here but there's a view all session uh here button here you can like go in to actually see like more of the
[15:16] uh certified assets in your organization. Uh there might be many. So that's where like the search bar and the uh filter come in handy. So for example here I can search for things that are
[15:28] relevant to revenue uh and then I can um limit to like just tables and uh if I click this uh asset it's going to actually take me like uh to the to the
[15:40] this table that like uh the user can take further actions. Okay, coming back to the discover page. Um below the certify uh session we have the uh trending session. Um so these are
[15:54] the assets that are gaining popularity uh recently across the organizations. Um so like uh this week is uh World Cup week. So we can see that like hey like the there's a genius space about like
[16:07] FIFA World Cup analysis that's really like getting popularity. And then below the trending section we have uh popular access uh assets section that like these are um like tables or or
[16:21] apps that are like popular in in your organization like uh over the past months. Uh and then below that we have the suggested assets. Uh as Steph mentioned these are like a personalized recommended assets uh based on the users
[16:36] past uh engagement. So like the system thinks that this might be uh interesting assets to the users. And then uh last but not least we have the uh glossaryy at the bottom that uh I will uh talk
[16:48] more like when we talk about domains. So yeah this is the uh overall layout of the uh discover page. Um and then uh let's look at domains uh which is a key
[17:00] part of the discover journey. Um so like uh the domains of the organization are listed at the top and then for each of the cars in the domains. Um so we can see there's an icon uh for the domains
[17:13] the name uh and then a short description and then also like the number of um assets in the in the domain that like give people an idea of uh the size of the domains. um like eight domains are
[17:27] highlighted in like this landing page but uh we can also click view all to see all the domains that's been cured uh for the organization. Uh and then uh we can also uh use the search bar to search
[17:39] like uh things that um are relevant. So like let's say I want to look at like the finance and um revenue domain. So let's go into the um this domain page to see what's in there. So uh yeah at the
[17:53] top there's uh like description of the domains that is um uh put in by the uh domain curators to like it could be like instruction or introduction of the
[18:05] domains. Um and then below we have uh a list of subdomains. Um so we do support like a hierarchy for domains. So like uh you can create subdomains under domains
[18:17] to like provide finer grain categorization and then below that we can see the uh assets that belong to these domains. Uh here we have like dashboards uh genius
[18:29] space and tables in these domains and then of course you can add like other things in the domains like apps or notebooks. Um they can all be added as assets to a domains. Um and then there's a session that shows like all the assets
[18:42] for this domains and then uh at the end we have glossary uh glossaryy that like belong to this domain and then like similar to like the landing page we can
[18:54] uh use search and also filtering to uh narrow down uh to find the um the data or asset that like I'm looking for. Like there's a filter panel here that like I
[19:06] can limit to let's say I'm only interested in dashboards that like uh let's say created by me and like they are certified and then like uh yeah that narrow down to like a smaller set of uh
[19:18] assets and then uh let clear the filters here. Um so uh let's also take a look at the subdomains. Um so we can just click subdomains and go in. Um the page layout
[19:32] for subdomain is actually like uh pretty similar to the layout of uh of these parent domains. It also like have description and then like the assets uh within the domains
[19:44] and then let's go back to the parent domains. Um I would like to also like to quickly showcase a glossary here. So go into glossaryy. Um you'll see like a wiki page type of u a page that uh
[20:00] actually like uh describes this uh glossary. Um so like it has like context about like who actually like owns this glossaryy. Uh and then like uh there's like a description that like talk about
[20:13] a definition or how to use it. Uh this can be consumed by both human and agents. Uh and then like uh the we can also like uh have like a relevance like assets um that are relevant to this
[20:26] glossaryy. And then uh at the bottom we also like created uh like a community type of experience that allow like people to actually post questions or comments uh about this grocery and and
[20:39] respond to comment. Okay. like uh this is like how um the uh from a user or consumer perspective like how like discover page can help them
[20:53] like find access through like domains or certify or like trending sessions and then uh next I'm going to demo like as a curator in the organization like how can I create domains or glossaryy and then
[21:06] also like how can I uh customize the layout and content of the of the discover page So I will start with uh creating domains. Um so like for like a curator
[21:18] who has permission to create domains um they can click create domains. Um domain underlying is backed by governance tax. So like if your
[21:30] organization is already using uh governance tags and like uh you could actually like promote certain like existing taxs into domains. So if you have like assets that are already like tagged by like the corresponding tag uh
[21:44] they will be automatically added to the domain. So like uh that makes like uh starting using domain simple. Um here I'll demo like creating a brand new domain. Um so like I just select like creating like a new tag. Um let's say I
[21:58] want to create a domain about support. Uh I can actually pick an icons and like a cutler here to make it like uh more visually appealing. So I can put in a
[22:10] subtitle let's say like customers support uh and then my description data metrics and access to
[22:27] customer support. Um and then we can put in some uh context here so like uh people can like know like if they have question like who they reach out to. Okay. These are all the metadata required for domains and then I'll
[22:40] create create. Okay. Um now the next step is actually like add uh assets to the domains. Um so I click uh adding assets. This will uh
[22:55] pop up the asset picker. Uh we can actually like add uh a few things together uh at the same time. So like let's say uh for support I'll add the incidents and like this
[23:09] support analysis. I can also like maybe pick a tables um and then add them to the domains and then now we can see like uh the three assets has been uh added to the
[23:22] domains. Um and then like initially like upon creation that domain this domain is actually in the draft mode. uh that allows the curator the opportunity to actually review it before like actually
[23:34] publishing it to uh make it available to everyone. So like now this looks good and then I'll just publish it and then uh yeah you see the uh draft uh
[23:46] tag is gone. So let's go back and see if we can find the domain we just created. Okay, here we go. We actually find it. So uh yeah, we created this domains. Um
[23:58] and then of course like uh over time we want to add more assets to the domain. We can do that by like uh uh here we can just click add to domains and then pop up the asset picker again and we can add
[24:10] more things to the domains. Um we can also like create subdomains for these domains. So like uh the flow is actually like uh pretty much the same as creating a domain. So let's say I want to add a
[24:23] domain called like case management here. Um so for interest of time I'm not going to fill in the other metadata. Uh but let's say create create a and then I can um add
[24:36] assets here for the subdomains and then uh it looks good. Click publish and then if we go back to the uh parent domain we can see that like a subdomain has been
[24:49] uh created under this domains to allow like finer grain uh categorization. Um and then like uh we can also create a glossaryy for this uh domains. So to do
[25:03] that we just click create a page. Um so I can specify like a name. So let's say I want to create a grocery called like uh customer um
[25:26] and then like I have um some content that I already created that I can just uh paste them here and then like this support like rich text. So we can actually like make the format like more
[25:38] beautiful for uh human consumers but like for agent they don't really care. Um so like uh we can also like add like additional uh related assets here. Um and then uh if this looks good, we can
[25:51] publish it. And then let's go back to the domains and we see that like uh grocery has been created for these domains and that can be consumed by both humans and uh
[26:04] agents. Okay. Um so I just demo like how to create a domain, subdomain and grocery. And then uh last I would like to demo like how to um customize the layout of
[26:16] the discover page. Um so like what you just saw is like the um the default layout of the discover page but different organization they may have different needs and then they want to like they want to customize uh the page
[26:30] to like uh let their users like find like easier to find like uh what they want and then we can also use the layout to uh uh promote like certain assets if if needed. So uh let's just uh create
[26:45] click like edit sessions. Um so there are a few uh customization that's allowed. So we uh support like creating a new sessions. Um or like uh you can
[26:57] also like hide an existing session or change the order of the session. So say we want to move like certified to be below trending like we can do that. Um so like uh you can do all these for all the um sessions. So here let's create a
[27:12] new sessions. So let's say there's a annual uh customer conference that's coming next week and then like our organization want to uh promote relevant
[27:25] data and assets. Um so I just create a new session called like annual customer c uh conference. Um and then like uh there are a few ways to um select as
[27:37] assets for these sessions. um we can define a search query. So if we define a search query then like anything that matches the search query
[27:50] will be automatically included in this sessions and then like you can also pin um certain assets to show up at the top. Um or uh if you don't want like uh the the the
[28:04] list of asset to keep increasing, you just want like some specific assets, you can also pick the manual mode which will allow you to just uh pick certain assets in this session. So let's say like here
[28:17] I only want like a couple like dashboards um like uh I want this uh >> hold up the screen. Oh >> the screen there. Okay.
[28:35] >> All right. It's good. Okay. Uh yeah, one just uh a few like customer specific like uh dashboards to show up there and also want like a let's say a genie space to be there. Um yeah, here I just pick
[28:48] four assets. So these are the four assets they will show up in the session. I can uh also change the order. So let's say I want the registration analysis to be at the top and then I move it to the top and then I click save. Um I have a
[29:01] chance to preview how the session looks like before publishing and then I click publish and here we go. We created a custom session for discover page and then whenever your users come
[29:13] to this page they will see like this uh customer conference session like uh near the top of the discover page and then let's say a few weeks later this conference ended. So like uh you can go
[29:26] in and you can uh just uh delete it and um yeah so uh that concludes the uh the demo. Um, so like I just showcase like as a curator how you can create domains,
[29:39] subdomains and glossaryy. And then also like how to customize the uh the layout of the discover page to uh make it a one-stop shop for your users to find uh
[29:51] what they what they need for their work. >> I'm handing it back to uh Steph.
[30:04] >> Cool. All right. Thanks. We have a couple of minutes left, so let's go through a little bit of uh what's next. So, uh yeah, Discovering Domains will be in public preview next week. So, we've been in beta for a couple of months now, but we're super excited that it'll be in
[30:16] public preview. So, um hopefully should be available to lots more more companies, more customers. So, please please try it. Um and we've got a really exciting pipeline of things that we're building on top of what you all just
[30:28] saw. Everything that you all just saw with the exception of the glossery uh is going to be in public preview next week. Glossery will be in private preview later this summer. Uh but there's a lot of other exciting things that we're continuing to invest in to make sure
[30:40] that both the curation of the discovery and domain experience continues to be faster and more efficient for you and also to make it a better experience for your end users. So one thing that we're investing in is we recognize that
[30:52] actually creating domains, populating domains can be heavy, right? And that's a lot of manual work. So we leveraging Genie what we're doing is we're actually identifying through some clustering algorithms uh what the what likely
[31:05] domains are that you should create. So Genie will recognize these we'll surface them for you. Uh we'll you know prefill all of the metadata for you. You can then review them and materialize them into proper domains.
[31:20] We're also continuing to invest in Genie powered discovery. What we hear from a lot of you is that, you know, point andclick is great, but a lot of customers love chat, especially these days. I just want to be able to describe what I'm looking for, and I want Genie to be able to go off and find it for me.
[31:32] So, we're continuing to improve the Genie experience to make sure that it covers all of those discovery needs really, really well. This includes embedding Genie into the discover page. So, you can see at the top here, if you ask a natural language question, you'll
[31:45] get a response um embedded directly into discover. Uh we're building a number of other things like the ability to summarize and ask questions about an asset if you're trying to understand if it's the right one for your use case. Um
[31:57] and um you know most importantly is uh we we are actively working on ensuring that all of the metadata that you create in the domains and the discover experience uh propagates to Genie. So
[32:09] Genie is aware of all the domains that you've created the metadata that you've added that you've added for those domains and so on and so forth. And then lastly, something else that we hear from a lot of folks and we'd love to get your perspectives is data
[32:21] products. So, uh, as you saw today, you can create domains, you can create subdomains, and you can add any asset within data bricks to those domains, subdomains, right? So, you can add tables, you can add views, notebooks,
[32:33] dashboards. But what we often hear from folks is that that's not enough, right? Sometimes you want to like bundle a certain number of things into one data product that you can then advertise as a proper data product that you can
[32:46] annotate as its own data product that has its own owner that uh you can provision permissions for um as a group and not just provisioning permissions as from an for each individual asset. So we're continuing to investigate how we
[32:59] might best bring data products as a first class object into the domain into the discovery experience. We've got a lot of other exciting things coming. Um, we we're we're actively
[33:11] working on making the discover in the domains experience account scoped. So, what we hear today is that there's a lot of pain involved in having to know which workspace something lives in, which meta store something lives in in order for you to be able to find it. So, we're
[33:23] working on making the discovery experience account scoped. Domains are already account scoped. So, you can add things to a domain from any workspace, any meta store within your account. But we want Discover to be the one-stop shop where you can find the high quality
[33:36] things from across your entire account. We also recognize that again like point-and-click UI might not be the best fit for everyone when it comes to creating and managing and maintaining domains. So, we're investing in programmatic management. Um, this
[33:48] includes, you know, through APIs and Terraform. And then similarly, we also want to make it such that you can leverage Genie code to create and populate any domains that you like. Um, we're also going to continue to
[34:00] invest in wiki- like documentation. So, you saw the glossery, we want to add something similar for domains so you can enrich it with additional context, additional insights and that that might be help uh be helpful for agents. And
[34:12] then something else that we're hearing a lot too is how can discover be my one-stop shop for discovery not just of data bricks assets but other data assets that I want my my users to find. So, we're we're looking into how might we
[34:24] link external assets and allow you to discover external assets through the discover page as well. These are just a few things that we're investing in, but hopefully from today you all were able to get a glimpse into what discover is,
[34:37] what domains is. Hopefully, it landed that they are in effect a super valuable tool not only for making data discovery and asset discovery easier for the humans of your organization, but also how they make up a critical component of
[34:49] the context stack that will make sure that any agents that you run on data bricks, any agents that you build on data bricks um are always reliable and accurate. So, with that, thank you. Uh
[35:01] we've got a QR code up here with a little form. Please scan it if you want to leave your your contact info. We love to talk to customers. So scan it if you if you feel comfortable. Leave your email. Let us know if you're okay with us reaching out to you and we'd love to grab time. We also have a couple of
[35:14] questions just to gauge interest in domains and discover and opportunity for you all to leave any feedback. But yeah, thank you so much.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.