UC Managed Tables: What's New and How to Decide, Adopt, and Scale

Summary

Unity Catalog managed tables now include Automatic Liquid Clustering, Auto-TTL, Auto Indexing, and Predictive Optimization, with benchmarks showing up to 20x query speed improvements and 50% storage cost reductions.
Catalog Commits enable writes from external engines and multi-statement transactions, while Managed Disaster Recovery provides cross-region resilience with 15-minute RPO and RTO targets for mission-critical data.
The new 'set managed' command simplifies migration from external tables, and a path-based redirect ensures existing path-based access continues working without breaking downstream pipelines.

UC Managed Tables: What's New and How to Decide, Adopt, and Scale

UC managed tables built on Delta and Iceberg table formats deliver faster query performance, lower storage costs, and automatic optimizations out of the box — and they've improved significantly in the last year. this video covers what's new, what UC managed tables unlock, and what you need to know to make an informed decision. We'll walk through the latest capabilities now available on UC managed tables. Highlights include Catalog Commits that enable writes from external engines and multi-statement transactions, Managed Disaster Recovery, Automatic Upgrades, Auto-TTL, Auto Indexing, Automatic Liquid Clustering and Predictive Optimization which can increase query speeds by 20x and reduce storage costs by 50%. From there, we'll address the questions we hear most and share best practices. How can I easily convert to managed tables if I access my external tables today by path? How can I segregate my data? How can I access these tables externally? You'll leave with a clear understanding of what UC managed tables offer today, what's changed, and a framework for deciding when and how to adopt them.
Talk By: Elizabeth Bowman, Product Manager, UC Managed Tables , Databricks ; Naga Raju Bhanoori, Staff Software Engineer, Databricks ;

Chapters

FAQs

What are the performance and cost benefits of Unity Catalog managed tables?

Unity Catalog managed tables with Automatic Liquid Clustering and Predictive Optimization can increase query speeds by up to 20x and reduce storage costs by up to 50% compared to manually maintained external tables. The Databricks Data and AI platform handles all table maintenance automatically, freeing data teams from scheduling optimization jobs.

What is Managed Disaster Recovery for Unity Catalog managed tables?

Managed Disaster Recovery provides turnkey, cross-region resilience for Unity Catalog managed tables, replicating data and metadata with 15-minute RPO and RTO targets. This video covers how Managed DR eliminates the need for brittle DIY replication pipelines while preserving full Delta and Iceberg table version history.

How do I migrate from external tables to Unity Catalog managed tables?

The 'set managed' command converts external tables to managed tables in place, and a path-based redirect ensures existing path-based access continues without breaking downstream pipelines. This makes migration straightforward for teams that currently access data by storage path rather than catalog reference.

What are Catalog Commits and why do they matter?

Catalog Commits allow external engines and tools outside of Databricks to write directly to Unity Catalog managed tables and enable multi-statement atomic transactions. This extends the full benefits of managed tables — including automatic optimization and governance — to polyglot environments where data is written from multiple engines.

Full transcript

[00:08] Hi everyone. We appreciate you coming here today for our talk on Unity catalog managed tables. Managed tables have come a long way in the last year and there's a lot that you can do on managed tables that you couldn't actually before. And
[00:21] whether you in this audience are all in on managed tables or you're just starting to take a look at them for the first time, we have something here for you today. But before I dive in, I want like to introduce myself. I'm Elizabeth
[00:33] Bowman. I'm a senior product manager working on Unity catalog manage tables. I also cover storage size observability and I'm very much into dance outside of work.
[00:45] This is Naga. He's our staff software engineer and he's really the brains behind managed tables and all the great data intelligence that makes make them work. You can also speak to him about open table formats, out of the box
[00:57] observability, as well as um soccer is something he likes outside of work. So, a quick show of hands. Who here has had to deal with a small file problem,
[01:09] schedule optimization jobs, or clean up storage after a migration? Or anyone on your team who's had to deal with this? Okay, hands up higher. Okay, lots of hands. Keep them up. Keep them up if you
[01:21] enjoy doing all this work. Exactly. And that's exactly my point. Nobody got into data engineering because they wanted to babysit their tables. We got into data engineering to build
[01:35] things to ship products and not to spend our Friday afternoons tuning false file small fall f small files or writing cleanup jobs. And also as more AI agents
[01:47] start quering your t to your tables or writing to your tables um that's going to make all that maintenance even the more harder. The whole idea behind managed tables is that all this babysitting work should be the
[02:00] platform's job and not yours. So what really is a manage table? There's really that one difference between Unity catalog managed tables in and unity catalog external tables from a
[02:13] functional perspective and that's not about who owns the data. So both in the case of external tables and in the case of managed tables you own the data. It's in your cloud storage account in your
[02:25] files and you own that data. The difference is about who does the table maintenance. With an external table, you have to take care of all that babysitting work. Whereas with a managed table, data bicks take care takes care
[02:37] of all that for you. To be clear, even though data bicks takes care of everything for you for a managed table, what if you wanted to tune it in a certain way that you wanted to, you can still do that. It's just that by default, manage for managed tables, you
[02:50] don't have to worry about all of this. Um, and now you might think, okay, so manage take tables take care of everything for me, but how does data bricks do this automatically but also
[03:02] safely? Think about it this way. Imagine you've got a car. Would you give your car to your 8-year-old kid to drive? No. Cuz your 8-year-old kid, they're so short, they can't see over the steering
[03:15] wheel. They can't see through the windshield, see the cars that are going past. you'd want to give it to an adult that both has experience but also has more visibility through the windshield. And managed tables are the same. It's
[03:29] very important because managed tables can actually see everything that's going on. And that's what makes us able to do these automatic optimizations safely and accurately.
[03:45] Manage tables are accessed by name catalog. schema.mmy. That sounds like a small detail, but that's actually what makes manage tables work the way they do. This is what allows Unity catalog to reliably see
[03:57] every read and write happening to that table. So, we can get a full picture. And that that visibility, like I was saying earlier, is what gives us the ability to safely and accurately optimize your tables. We can see how
[04:09] your table is being used. We can apply the right file layouts, um, file sizes, clean up old data, um, on the right cadence. So, we talked about all this and how manage tables are automatic, do
[04:21] all these great things for you. You don't have to babysit them like with external tables. But what benefits do these automatic optimizations actually give you? They give you 20 times faster queries
[04:33] and up to 50% higher co lower costs. Um, and in addition, it's worth noting that even if you tuned an external table perfectly, the managed tables still have a higher ceiling. they're still going to
[04:46] perform better because there are some optimizations we're able to do automatically when the catalog can see everything that's happening um that external tables can't. So for example uh we have metadata caching which Noug will tell you more about today and also with
[04:59] external tables you may have to run optimize or vacuum jobs on a regular cadence whereas manage tables are smarter. They think about does it make sense to run this job at this time.
[05:11] Don't take my word for it. I want to share a quote from a real customer. Zeppto has been able to save $1 million annually on storage costs by using Unity catalog manage tables and it's
[05:23] elimmitate eliminated the need for tedious manual effort on a daily basis. Many other customers are benefiting from manage tables too and in fact it's the
[05:35] majority table type on data bricks today. We have over 67,000 accounts using manage tables actively and we've also optimized 130 uh plus pabytes of data that we're optimizing continuously
[05:48] every month. I think also what's important about this statistic is not only the fact that we're optimizing a lot of data but we're doing so in a cost-effective manner. So we're figuring out what data should be optimized versus
[06:00] what might not be optimized. for example, if a table is too small or maybe it's not active, maybe it's not actually worth uh running those optimizations in those cases. So, we do it in a cost-effective manner that give you a return on investment. As you can
[06:13] see here on the right, we save customers $1 billion annually uh per year using managed tables. Um, so how does the platform actually pull all of this work off? Noga is going to show you how and dive into more detail.
[06:38] So, so how do we actually do this? Think of predictive optimization as a continuous feedback loop. It starts with what Unity catalog can see. Every read and write to a managed table flows through the catalog. This gives us
[06:50] visibility into the table layout, statistics, and how the table is actually being queried. From there, an AI model analyzes these signals, decides which optimizations are worth running. That part is really
[07:03] important. It isn't blindly running optimizations on a schedule on all the tables. It weighs the expected benefit against the cost and identifies the relevant tables and optimizations to
[07:15] run. These optimizations then are performed automatically in the background. Now, as your data and query patterns evolve, predictive optimization continually observes, learns, and adapts
[07:29] to keep your tables fully optimized over time. These are some of the automatic optimizations we run on manage tables. Some are focused on performance, helping
[07:42] your queries run faster. Others are focused on reducing storage costs. Let's start by looking at the optimizations that help your queries run fast. Starting with automatic liquid
[07:54] clustering. Predictive optimization monitors your usage patterns to understand how your data is being queried. Automatic liquid clustering evaluates different
[08:07] combination of column layouts and selects the clustering strategy that delivers the greatest improvement in query performance. Now once the clustering column selection is made, it incrementally reorganizes the data in
[08:19] the background, allowing queries to scan less data and run faster. But scanning less data is only one half of the story. File sizes are equally important too. Too many small files can
[08:32] slow down your queries. And now the optimal file size can vary based on size of the table and how it is being used. So predictive optimization continuously compacts small files and keeps file
[08:46] sizes optimized as the table grows in size. Now as your query and data continue to evolve, automatic liquid clustering keeps up with them. It continuously adjusts the layout in the background,
[08:59] ensuring your tables stay optimized even if your workloads change. Let's talk about automatic statistics next.
[09:11] Stats are like GPS for a query engine. They give the engine a quick snapshot of your data such as table size, row counts, the range of values in each column, and this information engine
[09:24] takes into account to take smarter shortcuts, choose better join strategies, skip irrelevant files, as you can see in the image on the left, and ultimately run queries much faster.
[09:37] Predictive optimization keeps these statistics up to date in the background by analyzing your workload patterns. It can also identify important columns that may be missing statistics and
[09:49] automatically collects them. This ensures the query engine always has the information it needs to consistently deliver fast performance. Elizabeth was mentioning about metadata caching. This is one of the advantage
[10:03] with managed tables. Instead of repeatedly reading metadata from cloud storage every time you run a query, databicks maintains an up-to-date view of the table, helping queries execute more efficiently at scale. Now, this is
[10:18] possible because Unity catalog reliably sees every right on a managed table. It allows us to maintain an accurate picture of this metadata, reducing the metadata overhead and improving query
[10:30] performance. Now let's talk about some of the features that help save you on storage costs. Automatic vacuum
[10:43] updates and deletes on your tables. They tend to leave old unreferenced data uh that continues to consume storage and are no longer referenced by your table or your queries. Predictive optimization
[10:57] continuously evaluates the amount of reclaimable storage and automatically runs vacuum when the expected savings justify the cost of the optimization. The result is lower storage costs
[11:10] without wasting compute on unnecessary maintenance. On the right we have cleaning up data after dropping a table. Dropping a manage table does not immediately delete
[11:22] the underlying data. Instead, the table remains recoverable with an undrop command for a configurable recovery window of up to 30 days. Once the recovery window expires, datab bricks
[11:35] automatically cleans up the underlying data files for you. Now, for an external table, dropping the table just removes it from the catalog. So, the underlying files actually remain in your cloud
[11:48] storage and require manual data change, manual cleanup. Many customers don't realize these files are still there. As a result, these orphan data can quietly accumulate into
[12:00] terabytes and sometimes pabytes of storage, driving up costs long after the table has been dropped. Now, the great thing about all these features is none of it is scheduled or
[12:13] tuned by you. It all happens automatically. No babysitting required.
[12:25] Now a natural question you might have is how do you know all these features are actually working? Sometimes the right answer is to optimize a table. Sometimes the right answer is to just leave it alone. Either way you will want visibility into these
[12:37] decisions. catalog explorer you can see not only the operations that are run on the table but also the optimizations that were evaluated and intentionally skipped with the exact reason why for example a table
[12:51] might not be large enough to justify clustering or as shown here predictive optimization evaluated the table and determined that the current layout is already optimal so no action was actually needed. This gives you
[13:05] visibility into how predictor optimization is managing your tables and the reasoning behind its decisions as well.
[13:19] Now let's talk about new features with manage tables this year. Starting with automatic upgrades. Databicks is constantly introducing new table features that improve performance, reliability, and cost efficiency. The
[13:33] challenge is that adopting them at scale requires understanding version compatibility, which clients are accessing what tables, and coordinating upgrades across thousands of tables. As a result, many
[13:45] customers miss out on some of these capabilities. For example, only about 20% of managed tables today use road tracking and about 10% use checkpoint v2. And maybe some of
[13:57] you are hearing about those terms for the first time. Uh, automatic upgrades does all the work for you to keep your tables up to date. It continuously verifies compatibility and
[14:09] automatically upgrades your tables with eligible features, making sure they benefit from all the latest innovations.
[14:21] One of our customers, Kovitricus, put it really well. Audrey, who leads data platform engineering there, told us that the biggest benefit isn't just the feature themselves, but isn't it's no longer having to keep track of every new
[14:33] table feature on every table out there.
[14:45] I'm excited to talk about this one. Automatic liquid clustering is now enabled by default. Any newly created managed table will get this capability by default. And combined with predictive optimization, you no longer have to
[14:57] think about how your data should be laid out to get good query performance. Moving on to some more new features, autoTL.
[15:10] Many data sets have a natural expiration date. Instead of building maintaining custom deletion cleanup pipelines with this feature you simply specify a deletion timestamp column and a
[15:23] retention period autotl automatically identifies expired data, deletes it and reclaims the associated storage. This keeps the data sets lean storage costs
[15:36] low without the need for any custom cleanup pipelines.
[15:48] Let's talk about zero next. Traditionally getting data into the lakehouse when standing up managing an entire injection stack, message buses, injection services, and all the infrastructure that comes with them.
[16:00] Instead, Zerobus provides a push-based API to write directly into your manage tables. Whether it is click stream events, transaction data, or other application data. Your applications can
[16:12] send data using familiar protocols, rest, gRPC, open telemetry without the need for any message buses or injection pipelines. Data bricks will handle scaling, durability, and any storage
[16:25] optimizations behind the scenes. And now this isn't just about simplicity. Zerobus scales from small workloads to pedabyte scale deployments. It can consistently provide 12 GB per
[16:38] second sustained throughput while maintaining the same straightforward uh injection experience.
[16:50] This code here captures exactly what Zerobus is designed to deliver. Joby Aviation is ingesting gigabytes of telemetry data per minute from its manufacturing sites. But the key takeaway for me isn't the volume of the data. It's that they keep adding more
[17:02] sites, more devices, and more data. And they don't have to redesign their architecture. Zerobus simply scales with them.
[17:18] Let's talk about Lakehouse indexing. Many analytics data sets contain large volumes of text. You might be searching logs, support tickets, product descriptions. Finding the right record sometimes can be like finding a needle
[17:31] in a haststack. Let's look at this example. You have a logs table and the query here is searching for a particular string in a message column. Today that query may need to scan the entire row,
[17:43] entire table to find the matching rows. Full text search indexes solve this exact problem by building a specialized index over text columns. So you can simply create an index on the message
[17:56] column, continue using the same exact SQL and the query engine automatically uses the index to quickly narrow down the rows you're looking for. This feature is currently in beta. And
[18:09] looking ahead, we plan to support additional index types and over time predictive optimization will automatically create these indexes for you.
[18:25] So far I've talked about many existing and new features that manage tables optimize for performance, storage efficiency, operational simplicity. But what happens when an entire region goes down?
[18:37] Manage disaster recovery protects your lakehouse by continuously replicating Unity catalog workspace assets and your data to a secondary region. The experience is designed to be
[18:50] seamless. There are no configuration changes required. Applications can continue connecting through the same stable URL endpoint during the failover and fall back as if nothing happened.
[19:02] This is possible because all rights flow through the Unity catalog and each table maintains a unique identity across time. This gives data bricks a consistent view
[19:14] of every change allowing us to keep the data accurately synchronized across regions.
[19:29] Now let's talk about transactions. Historically, organizations running transactional workloads had to maintain separate data warehouses alongside Lakehouse because the lakehouse architecture lack native transaction support with managed tables. That's no
[19:43] longer the case. Databicks now supports multi-statement multi-table transactions with rowle concurrency on open table formats. In this example, we're updating
[19:55] skew 25 across two different tables. Multi-table transactions ensure those changes are committed together or not at all. And it's not just limited to two tables or a single statement. You can
[20:08] bundle multiple operations into a single transaction bringing transactional workloads directly into the lakehouse. Now that I've talked through bunch of latest features available on manage
[20:20] tables, Elizabeth will talk about how manage tables are open. >> Thanks Naga. So many exciting new features on managed tables, it's hard to even keep track of all them.
[20:32] I wanted to emphasize the fact that managed tables are open and interoperable. First of all, managed tables are open. So open table formats, whether that's delta, whether that's iceberg, and as we
[20:46] mentioned earlier, this is in your cloud storage bucket account that you own. You also get the same benefits whether that's predictive optimization, automatic upgrades, all the great things that Naga told us about no matter what table format that you're using. We're
[21:00] also excited to announce at this summit that managed iceberg tables are generally available. So unlike before, what that means now is that for managed icebury tables, you can use vector
[21:12] search, data profiling, sync to lake base, and we also have automatic change data feed, uh, which is a new feature. It looks like, oh, it looks like there's like a question in the audience. Um,
[21:32] >> that's a great question. Thanks, Jeff. Um he wanted to know if he could write to a managed table from open-source spark. Um and actually that's a question that we get very often is can you access your managed tables from other tools and the
[21:47] answer is yes. Managed tables are even more interoperable today than they have been in the past. So first of all you probably might already know you can read managed tables from JDBC using JDBC OBC
[22:00] delta sharing open APIs. So you can read manage tables from anywhere but also at this summit you can also write to manage tables from a wide ecosystem of different engines. Um and what that
[22:13] means is not only can you write from the iceberg um uh rest catalog now but you can also write from using the unity catalog open APIs. Um so like Jeff said
[22:25] it's not just about read but it's also about writes too. So managed tables are really fully interoperable now at this summit. Catalog commits which is now generally available is actually what
[22:38] makes this possible. It's an open standard that that makes the catalog the commit coordinator. The same way for Delta as has been always the case for iceberg. So really excited about this this change in this summit. This is how
[22:52] PepsiCo put it. They've empowered their teams to use preferred tools with managed tables. all while maintaining governance for a truly interoperable experience that works across multiple
[23:04] compute engines. Beyond interoperability, this governance piece is actually a key point because catalog commits not only expand interoperability when we're talking
[23:16] about rights from all of your different engines and tools, but catalog commits also ensure that governance applies across all of your engines. So let's say that you want to mask certain email addresses with PII data or um for
[23:29] example you want to filter out certain rows in GDPR region maybe related to the European Union you can do that by applying a policy once in Unity catalog and that is respected across all of your
[23:41] engines inside data bricks across the whole wide ecosystem. So again you get fine grain governance across all your engines.
[23:53] So, we've talked about managed tables. We've talked about how they can give you faster queries, lower costs, you don't have to babysit them, less manual effort, so you've saved a lot of time. Um, and they're open and interoperable. So, you might be thinking like, that's fantastic, but how can I actually get
[24:07] there? We have an easy conversion command that makes this process very easy for you to move from unique catalog external tables to unique catalog manage tables. Um you just run this command and your table is
[24:20] converted. U some benefits of using this command. It it retains table history. It maintains uh configurations whether that's your table name, settings, permissions, views. It also helps minimize reader writer downtime. So on
[24:33] the order of minutes or eliminated entirely depending on what data bricks runtime version you're on. And also it handles concurrent writes during conversion. So as you're converting your table, you can keep writing to it and we'll take care of that all for you in
[24:45] the background. In addition, what's new new this year is um path-based redirect. And so customers have mentioned sometimes that hey, I have all this path-based code changing to be name based that's too much effort. I don't
[24:58] even know where all that lives. Like so like that would be a nightmare. And so we've made this even easier for you with this new feature. So any pathbased code in the past will redirect that to the new managed location for you. Um so you
[25:10] can either update that to name based access on your own time or if you didn't want to update it at all, that's also okay. So that takes care of any path-based access from the past and then moving forward use name based access. So again the catalog can see everything that's happening.
[25:24] In addition to the set managed command which is very easy is um this easy conversion in the user interface within catalog explorer. If you go in catalog explorer to a specific table under about
[25:37] this table you have table type it will say manage external um etc. You can hover on that table type and click on it and that will give you a screen that talks about the different table types and the differences between them, where
[25:50] you're at, what else is available. And if you'd like, you can also press continue and that will take you to a workflow here where you can convert your managed table or your external table to be managed. And so you can either select
[26:02] a specific table or you can also do this for an entire schema. So you can select all the tables within the schema and convert that to be managed. um that will automatically then create a notebook for you with that uh syntax of that set
[26:14] manage command. It'll have that roll back command in case you need that. Um so within 40 days you can always roll back in case you were like oh I changed my mind I just want to undo. Um you have that optionality. We have more information there in the notebook as
[26:26] well as best best practices links just to make this process as seamless as possible. Um ASOS has used the set manage command. They've converted 278 terabytes of ma uh two managed tables.
[26:39] They said the process was smooth uh non-disruptive, straightforward and post conversion they've realized lower storage cost, faster queries, far less manual maintenance and really the effort uh to value ratio has been excellent. So
[26:53] they've really gotten a return on investment here. It's been an easy process to move to manage tables and they've saved a lot of money here. Um, so again, you know, you just run that simple command and then you're done. I think another point to mention is that
[27:06] customers have run this command on tables that are up to 400 uh 400 terabytes in size for one specific table. So this command has been built to work on tables no matter what the scale.
[27:24] Yeah. Um the other one was the stage question. So questions and you'll you'll come to the front, but thanks for uh being engaging. Um and uh this this next slide I did want to mention this important point which is hey when I create a managed table or let's say I
[27:36] convert to a managed table a common question that we get is what's the location where that table is created as you can see the syntax is is different here with a manage table doesn't take a path whereas external table you see when
[27:49] you create it it actually takes a path um and in both cases as we said earlier that that file is created in your cl own cloud storage account and a file path that you can see. So you can just go in catalog explorer or describe extended to
[28:03] find that path. So you can go there any time. You you we also have a way for you to look up in bulk all the list of your table names and paths if you should so want that. Um but why do we have this difference here in how these tables are being created? The difference is that
[28:16] with managed tables you get data independence. So you've got your storage and you have your application code. right now. Um, before manage tables, they're tied together and that's actually not a good thing because what
[28:28] if something about your storage is changing? That means that your workloads then break. So, that's not something that you actually want. And what's great about manage tables is it decouples the two. So, same as how we've decoupled compute uh and storage in the past,
[28:41] that's provided you flexibility, the ability to scale, similar here. So, this gives this makes the process easier, but it also reduces kind of these issues that are here kind of for no reason that they didn't need to be. Um, and so
[28:54] that's one aspect. And I think another point is with external tables, it's worth noting that if you create a table and then you drop it and you recreate the table with the same name and then the same path over the course of time,
[29:06] there are some cases where this can cause data corruption. So that's not ideal. And also cache information about your table is also no longer linked. So there's some aspects with external tables that aren't as great. We've
[29:18] solved that with managed tables by making sure that whenever your managed table is being created is created with a unique file path. So that again gets back to some of the design decisions between these differences of manage and
[29:31] external tables. Um and so okay I've told you about all this right and a lot of customers they create their manage tables they're creating their meta store they may use you know cataloges or schemas logical ways to separate them uh
[29:45] and user based access controls to handle GDPR um requirements but hey there are some customers that are that say well I want to physically uh segregate my data within my cloud storage I want these
[29:58] managed tables over here and those managed tables over here we're excited to announce at this summit that that we have this new command which is available and now generally available today and this is uh set manage location. So what
[30:11] you can do is before converting your table to be managed you can run this command to set a path at the schema level or the catalog level and we'll respect that. So when you we convert your table to be managed that manage
[30:24] table will be be created in that physical place. So in that sense, it still gives you the capability to physically organize your data and your storage as you want with manage tables. Whether that's you want to separate them for cost attribution purposes or maybe
[30:38] regulatory compliance reasons, you still have that full capability today. As One Trust put it, with Unity catalog managed tables, our data stays in open formats in our own cloud storage and we
[30:52] can always look up exactly where the tables files live. We keep full ownership and control while data bricks handles the optimizations for us.
[31:07] So what's next on the road map? We have automatic storage turing or archival. We're looking into ways to save you on cost by reducing by bringing your data to lower cost tiers where it would make sense to do so. We're also
[31:19] going to have a multi-level name space in Unity catalog moving forward so that you've got like catalog schema. You can have a schema under that schema and then you can have your table. So more more granular control in the future and
[31:31] that's also important for customers that do need to physically separate their manage tables and storage. This gives you a little bit more flexibility from that perspective. So that's coming soon. And then also another one is storage size observability.
[31:44] This is a huge actually pain point customers have had is what is the size of my storage not just for my tables but for a specific catalog for all of my schemas what all compared to each other
[31:56] they want to see the size for example across different domains ac across different products a different across different customers um how much storage is is being taken up for each um and that's important when it comes to storage costs and keeping that under
[32:08] control. Um, so you'll be able to see all of that this information whether that's for a table, a schema or a catalog. And then also you can see growth over time. So how is my storage size growing? So some customers have
[32:21] said, "Hey, I had this table and it was growing completely out of control and I didn't even realize it until I I took a look, you know, in my in my storage and this is not very visible today." But with this new user interface, this this will be more visible. We're also going
[32:33] to be looking to add recommendations. So for example, what if there's a table that's inactive? you know, you don't query it often. Maybe you should actually delete that to save on storage costs or maybe you have duplicate tables
[32:45] or maybe your retention window is just too long for certain tables and you actually could shorten it. Um maybe that would reduce the amount of time travelable data but you don't need that and that helps save you on storage costs. So we're really looking forward
[32:57] to this um having this on the road map and this is going to be under um something called the governance hub. We have another session on this which I'll I'll point you to as well. So bottom line, no one likes to babysit
[33:10] their tables. The bottom line is that managed tables manage themselves. They give you faster queries, lower costs, they help save you time and
[33:22] manual effort all while being open and interopable across the entire ecosystem. And it's your data. Again, manage tables manage themselves.
[33:35] Thanks everyone for coming. Here are some other sessions that you can dive into if you want more information. We've got your guide to open table formats. This one is more about catalog commits. Dives more more deeply into that. Uh at
[33:48] 410 p.m. we've got observability in the AI era. This is the governance hub. So that's where the storage observability part of it will live. Uh in the future that's again later today. We've got a
[34:00] session on where um AI governance is headed. So um Unity catalog not only governing all your data will also govern all your all of your AI agents and as you have more more AI agents working on
[34:12] your data unity catalog managed tables are great for that because they keep all of your data automatically op optimized. Uh we also have a session on interoperability with Unity catalog. So that's reading cal your reading your
[34:24] managed tables or writing to your managed tables like Jeff mentioned uh more information on during this session. And then we also have one that's a deep dive about automatic liquid clustering, intelligent storage and all of that um
[34:36] part of it on Thursday. So thanks a lot everyone for coming. Um you can come up to the front for questions afterwards. Uh we really appreciate your time. So again, manage tables manage themselves. Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.