Data Intelligence on Unity Catalog Managed Tables Powered by Predictive Optimization

Summary

• Predictive Optimization: Databricks automatically optimizes Unity Catalog managed tables with intelligent data layouts, improving query performance and reducing storage costs.
• Automated Table Management: Features like liquid clustering, automated statistics, and intelligent vacuuming eliminate manual table maintenance while optimizing performance.
• Real-World Impact: This video shares customer examples and observability features showing how automation drives measurable performance improvements and cost savings.

Data Intelligence on Unity Catalog Managed Tables Powered by Predictive Optimization

Talk by Cindy Jiang, Product Manager, Databricks ; Naga Raju Bhanoori, Staff Software Engineer, Databricks
In this video, we’ll explore the data intelligence capabilities within Databricks, focusing on Predictive Optimization. This feature enhances the performance of Unity Catalog managed tables by automatically optimizing data layouts, resulting in improved query performance and reduced storage costs. You’ll learn how Predictive Optimization works and see real-world examples of customers using it to fully automate data layout management. We’ll also share a preview of the exciting features and enhancements coming down the road.
Here’s more to explore:
Unified and open governance for data and AI: https://www.databricks.com/product/unity-catalog
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

Chapters

• 0:00 Introduction to the platform
• 1:09 Challenges in data management
• 5:41 Automatic liquid clustering
• 14:46 ROI and query performance
• 19:41 Automating table statistics
• 24:06 Intelligent vacuuming
• 30:33 Adoption and customer success
• 32:43 Observability and insights
• 36:03 Roadmap and conclusion

FAQs

What is Predictive Optimization in Databricks?

This video defines Predictive Optimization as a feature that automatically optimizes data layouts and table management in Unity Catalog managed tables. Using AI-powered capabilities, it eliminates the need for manual table maintenance decisions while improving query performance and reducing storage costs.

What data management tasks does Predictive Optimization handle?

This video explains that Predictive Optimization automates liquid clustering for optimal data layout, automatic statistic generation and updates, and intelligent vacuuming operations. It removes the burden of deciding which columns to cluster, when to run optimize, and how to maintain table health.

How do customers measure Predictive Optimization impact?

This video covers observability and insights features that help users understand performance improvements and ROI. This video shares real customer examples demonstrating query performance gains and cost efficiency improvements achieved through automated data layout management.

What challenges does this video address?

This video illustrates challenges data engineering teams face: how to lay out tables, apply clustering correctly, determine optimal file sizes, decide on maintenance frequency, and select which optimizations to run. Predictive Optimization removes this complexity by making these decisions automatically.

Full transcript

0:00 Introduction to the platform

Hi everyone. Shall we get started? So we're super excited to have all all of you here today with us. In this talk, we'll cover how data bricks brings data intelligence to your UC manage tables. This is all possible because of predictive optimization. We'll first give you a broad overview of what problems we're trying to solve with data intelligence, followed by a technical deep dive into how predictive optimization works behind the scenes.
We'll then share how you can understand all these AI powered capabilities with observability and end on a sneak peek into our upcoming road map. Before we get started, just a little bit about myself. I'm a product manager here at data bricks working on the open lakehouse as well as how we can bring data intelligence to your tables and I have Naga here with me today. He's the technical lead of predictive optimization and one of the smart brains behind bring all these awesome data intelligence to your tables.
Okay, before I get into the details of predictive optimization, let's take a step back and understand what problems we're trying to solve. So imagine you're the head of data engineering at a large company. By the way, I actually saw the signups for this talk. I know that in the audience today here with us, we actually have quite a few head of data engineering at large companies with us today. So as a data engineer, you have to manage many tables and many data sets.

1:09 Challenges in data management

You have to manage multiple pipelines and to generate insights from your data which is growing every day. And this is why data management has been a very hard problem. We all know that proper table maintenance can improve query performance and cost efficiency. But getting this right requires technical expertise, manual overhead, and continuous adjustments as your business and data use cases evolve. Folks here are probably relate to some of these questions that data engineering teams have to figure out.
How do I lay out my table? Should I apply any clustering techniques? If so, on which columns? What are the most optimal file sizes? How when should I run maintenance? And what optimizations do I run? Do I run vacuum? Do I run optimize? Oh, but hold on. On which tables should I be optimizing? And actually, if I want my queries to go fast, what do I do? Do I need to update stats? And if so, on what columns? And once again, for all these optimizations, how frequently should I be running them?
And on what tables? Now with all these questions on your critical workloads, you probably wish to have an army of data engineers tackling these problems for you. And once these questions are answered, you have to manage the operational overhead of actually running these optimizations such as through scouting jobs, looking through failures, and managing the underlying infrastructure. And this is not just a one-time setup.
You have to continuously update these jobs when your data grows, new tables are added, and access patterns change. As data and AI use case have exploded over the past few years, we've heard from so many customers that just not able to keep up with optimizing tables created by expanding business needs. But thanks to our data bricks data intelligence platform, we simplify all the data management for you with predictive optimization.
So we use advanced AI models to determine which tables to optimize and data bricks will automatically perform these optimizations on your UC manage tables on your behalf. And the operations we support includes compaction vacuum automatic liquid clustering and automatic statistics. And how do we do that? So thanks to the catalog we actually have all the reads and write access captured through the unity catalog and we can understand exactly how your tables is being used and its characteristics.
This is why with Unity catalog manage tables together with predictive optimization. We can take care of all the data management for you and let you focus on extracting the business insights. You no longer need an army of data engineers just to closely monitor all your tables to run optimizations. And I'm super excited to share that predict optimization is widely available everywhere on manage delta, manage iceberg, and on our declarative pipelines.
And so this all sounds really great, but you might be thinking, how do I turn it on? So you have a new account created after November of 2024, you're all set. We have predictive optimization enabled by default for you. If you have an existing account, go to the account console today and toggle predictive optimization as enabled. And if you want to wait, we're actually rolling out predictive optimization by default everywhere on the platform as well.
And note that predictive optimization is only available on UC manage tables. So make sure that you're using UC manage tables in the first place. And what is the net effect? With predictive optimization enabled, you'll see your queries running faster, even up to 20 times. You also see the total cost of ownership going down as predictive optimization can have your storage cost. Now, let me pass on to Naga to give us a deep dive into how predictive optimization works behind the scenes.

5:41 Automatic liquid clustering

All right. Thanks. So in this talk I'm going to use an example table a workload and show you how predictive optimization works in action as this table keeps growing in size and gets queried. Okay, let's take this sales table I crafted. This has information of orders coming in each day. I'm showing two columns here. Date column, customer ID. Date column indicates the day the order was placed. Customer ID is a customer who placed the order.
Uh let's see. Sales price tells us the cost of each order. There are more columns in there that are not not shown right now. Each day data gets ingested in different batches. For example, we have some data coming in on June 10th. Some more data coming in on June 11th. Let's say for the data that came in on June 10th, we have that in five files. I've put them file one through file five. Now, what are those alphabets I put in there?
File one here says a parking file on the table. ABC shows that this file has data for customers A, B and C. Um, similar to June 10, June 11 has another five files land in here without say any data layout optimizations. This data right now is stored in what I call injection order. Now each file across the table has information about multiple customers. Okay, that's the table setup. Let's say we run this query on this table to find out the total price of all orders on June 10th for a particular customer.
A pretty selective query. So what does the query engine do next? It needs to figure out for this query what are all the files to consider and what are the files to look at. So what the query engine does is it leverages data skipping stats minmax values of these predicate columns on table metadata to help narrow down the files to read. Now narrowing down this number of files to read is actually critical for query performance.
It reduces the total amount of data that's scanned by the query. Okay, so I've laid out all the 10 files here with different minmax values. Let's go ahead apply our predicate the date. That's great. So we get to ignore everything from June 11th just by looking at the minmax value of the column date. But what about the files on June 10th? If you look closely, we would end up considering all the files on June 10th to answer this query.
Why? If you look closely the file two for example the minmax values are A and C. So technically it can contain information about customer B although in reality it doesn't. Now such queries at scale can get expensive and they read unnecessary data from cloud storage and also slow down your queries. So the challenge really here is that data isn't colloccated or clustered here say by customer ID. In this case it's kind of scattered all over the place.
At the end of the day we're just getting a pruning rate or of 50%. So that is half the files eliminated from consideration. Can we do better? Luckily with predictive optimization we have automatic liquid clustering with data intelligence. How can you use it? It is super simple. You just need to set cluster by auto on any of your new or existing tables and you will get automatic liquid clustering with auto. Predictive optimization automatically chooses clustering columns, clusters your data and ensures your queries are performant.
How does it work? Let's take a closer look. Firstly, predictor optimization or in short PO captures telemetry across different types of workload. For example, you might have queries coming from streaming jobs or dashboards and they might span different sets of tables. All these queries flow through Unity catalog and that allows predictive optimization to understand your entire workload. That includes things like how many scans are happening, what are the sizes of the files that are read and that are getting skipped, which columns are used in your predicates, what are the predicate expression, everything about your queries.
Now across all the columns in your queries, predictive optimization leverages a workload model to carefully identify and narrow down to a candidate set of columns that actually improve the pruning rates of your workload. Basically reducing the amount of data red. So let's say the candidate set of clustering columns here are we still consider the injection order current layout. Sometimes that might just be good enough.
Or say date, customer ID, date, customer ID. And I included a few more columns. Date and ship date. It can be a bigger exhaustive list, but I'm just showing a smaller set here. Now given a set of these clustering candidates, predictor optimization generates a hypothetical whatif layout of your data and actually runs a shadow replay of your queries against them. Now why do we need to do this? We do this to get a higher degree of confidence in the clustering column selection to actually make sure it improves the query performance and the pruning rate across your workload.
Now PO doesn't fully rerun your queries. It replays the captured predicate expressions and evaluates how much data we would have pruned with each of these layouts and helps us choose the right clustering column that delivers the best pruning rate across your workload. Let's look at the resulting pruning rates. We knew the current layout for example here was at 50% and even if you're clustered by date would have been still at 50%.
Looks like if you had clustered by customer ID for this query pattern looks like it would have been at 80% and date and customer ID would have given us 90%. Pay attention to that date and ship date. This process actually allows us to detect any correlated columns in your workload. very likely the date the day order was placed and the day order was shipped are correlated and I'm sure your data has such correlated columns and pure would identify such columns because including and clustering both such columns isn't really uh isn't really going to yield much benefit to your query performance across these choices say we pick date customer ID as a clustering scheme and what predict optim optimization does next is applies this clustering scheme and leverages the optimize SQL command to actually go cluster your data.
Clustering reorganizes your data per the chosen clustering scheme. It is incremental. It will not touch data that's already well clustered. And for some reason if you have some small files in there part of the optimize command they all get compacted and clustered in goodsized files that's determined automatically based on your table size without needing any tuning whatsoever. We all know reading data from small files can be a killer for your query performance.
We don't need we need to avoid that. Now, predictor optimization applies this clustering scheme and does all of this heavy lifting only when we see the benefits from the scan improvements outweigh the cost of clustering. All right, we've say gone ahead clustered this table and now we rerun the query. It only reads one file and that's great. We got 90% pruning rate. Awesome. But your table right now is tuned only for your current query patterns.
But maybe query patterns evolve on some of your data. New columns get added. Business needs change and predictive optimization adapts to them. PO repeats this exact same process, changes clustering columns, adds new or removes old, and recclusters your data as needed. Now imagine the manual effort you have to put through to tune uh every table across your data set. That's really hard. And automatic liquid clustering can do this for you with predictive optimization.
Okay. So our data on these two days is well clustered but we have more data coming in every day and it's important for all of the data to be well clustered so the queries remain performant. So we need to run optimize at a certain frequency. But how often would you run optimize? Would you know the injection patterns for all your tables? Don't worry, predictive optimization does. Predictive optimization understands your workload and runs optimize automatically.

14:46 ROI and query performance

We capture telemetry from your workload. The telemetry predictor optimization leverages here is the amount of uncclustered bytes on your table. your table usage, how much how many times is your table getting queried, what are the sizes of files getting scanned and PO determines the return of investment or ROI to run optimize. Using this telemetry, PO estimates the following one obvious thing was the estimated cost of clustering.
The important thing is what is the estimated query speed up? And the speed up is from two aspects. One is improved data skipping with wellclustered data and the other is improved performance from reading goods sized files. Basically looking for evidence of if you have any small files in there and your scans are reading them. Okay, I have a set of four tables here with different access patterns and I wanted to walk you guys through what PO does in each of these scenarios and how it determines uh optimized to be run.
Let's look at table one. Looks like the table usage is pretty low. And for PO to consider this pretty low, you're barely touching this table. You're practically not using it. And even though and looks like the cost of clustering is is not cheap. Even though you have bunch of small files and the estimated query speed up is high, it isn't worth running optimize if you're barely touching this table. So for now, predict optimization determines the ROI is low.
Let's look at table two. On the other hand, it's getting used heavily and the query speed up is possible with minimal clustering cost. The sizes of the files scanned is moderate. So you have presence of some small files and the speed up you get is medium. But hey, it's pretty cheap to cluster. Maybe you have very little amount of data to cluster. So the ROI is deemed medium and PO would run optimize in this case. What about table 3?
The cost to cluster the data is really cheap but looks like the estimated query speed up isn't much and the data layout is good. The files are in ideal shape already. So there isn't really any point in running optimize for this table right now. So the ROI is determined low. What about table four? Even though the clustering costs are high, the usage of the stable is pretty high and the estimated query speed up is high.
A combination of both compacting data and wellclustered data. So we should pick this table and run optimize against it. So the ROI is high. So across this telemetry table two and four will be picked for clustering right now. But those other tables don't worry about it. Pure revisits those tables routinely and figures out if optimize is needed and runs optimize automatically in the background. Let's look at how customers reacted to this feature.
Customer branching minds closely monitored their query performance and saw noticeable improvements. For one of their tables, the average query times went from 100 seconds to less than a second. from Health Rise saw their query sign up to 10 times faster with automatic liquid clustering without lifting a finger. In addition to these amazing performance improvements, customers love that the heavy lifting of tuning and maintaining the data layout is now done by Auto Liquid efficiently.
It saves valuable time from all the manual manual work you have to put in to maintain and tune the data layout yourself. Customers like CFC Underwriting echo the exact same sentiment. Auto liquid and predict optimization completely eliminate the need for manual data layout maintenance. Lastly, we are really excited to see customers broadly adopting autolid tables enabling on thousands of tables in production environments.
That's amazing to see. It is super easy to enable and the best part is they have actually seen compute costs go down over time. Okay, now back to our example workload. I've shown you guys a rather simple query before. I'm sure your queries have that span multiple joins and columns etc. multiple tables and joins. So now let's look at a slightly more uh involved query. I have crafted this query. It's returning top 100 item warehouse combinations by total sales count in the last year and showing how many of those sales leveraged promotion and how many did not.

19:41 Automating table statistics

So I have a join on item and warehouse dimension tables to get warehouse name the description of the items and a left join on the promotion table to isolate sales with and without promotion. And lastly, date dimension to limit the orders to all of last year. Now for this query to be performant, it is important that we apply these joins in the right order. So the query actually is performant and reads less amount of data.
And not just the join order, it is important for the join strategy. Do we use shuffle join or broadcast join to make sure the query can run faster? Now query engines are actually smart enough to figure out and make these decisions correctly if they have the right information or statistics about your table. Now what are these statistics I'm talking about? So there are broadly two types of stats for tables that come into play.
One is the data skipping stats. I've briefly touched upon it earlier. These are minmax stats per column per file in your table metadata. They help with file skipping and reduce the size of the data read by your queries. And the second one is what we call optimizer stats. These are how many rows are in your table, what's the size of your table for each of the columns, number of distinct values, number of null counts, and they are used in query planning to pick the right join order and the right join strategy to make sure your queries are actually performant. analyze table command helps collect these statistics for a specified table.
I'm not sure how many of you here actually know about this command. At least from our data science, less than 2% of your tables actually run analyze on them. Now, now the takeaway is not for you guys to go run analyze. Uh predictive optimization can do even better. Automatic statistics is enabled by default with predictive optimization. Automatic stat statistics figures out the columns that need statistics and maintains them.
With auto stats, both optimizer and data skipping stats are collected and maintained automatically. Where are these stats collected? Stats are collected during writes on your tables. When you're writing data to your tables, they are collected during that time and with minimal overhead. Optimizer stats in specific are far more efficient to be generated during rides than running analyze later. And also if you have updates and deletes on your table, optimizer stats tend to go go stale and predict optimization detects that and refreshes these stale stats automatically for you in the background.
Also, predict optimization adapts to your changing query patterns, collects stats on additional columns based on your workload patterns if needed. The query I showed you guys before I ran it with and without auto stats, it went from order of minutes to order of seconds. It was 10 times faster. It just goes to show how the right join order and join strategy can dramatically improve your query performance. Okay, U that was about improving your query performance.
Now let's look at how predictor optimization can reduce your co storage cost in half. Back to our example table, we keep loading data every day. uh let's say the table grows in size and if your workload isn't purely append only and tend to it tends to have updates and deletes it can result in a lot of unreferenced files without you recognizing about it and they lead to additional storage cost. So how do we keep these storage costs in check? vacuum command.

24:06 Intelligent vacuuming

Vacuum SQL command makes sure it cleans up these unreferenced files and keeps your storage costs in check. How does vacuum work? Vacuum briefly does the following. It figures out it lists all the files for uh on your table by listing from cloud storage leveraging the table retention property on your table which is defaulted at 7 days. Vacuum identifies what are the valid files for your table to be kept, isolates these unreferenced files and goes ahead and deletes them.
So would you know what tables to run vacuum on and at what frequency? I know you guys can guess this. Uh predictor optimization understands your workload would automatically identify these eligible tables and run vacuum on them automatically. Similar to clustering let's look at how PO figures out which tables to run vacuum. So the telemetry leveraged here is from your commit patterns how many bytes are getting added how many bytes are getting removed what are their sizes what's the retention window of your table and how many files are present in your table using this predictor optimization estimates the amount of garbage collectible data or vacuumable data and what's the cost of what's the estimated cost to run vacuum All right, let's do a similar exercise.
I have four different tables here with uh different access patterns. Let's look at table one. High vacuumable data. So, the storage savings are high and the cost to run vacuum is pretty low. The ROI here is high. We should run vacuum on this table. And you can see for table two and four, there isn't that much garbage collectible data compared to the cost of the vacuum. So no point running vacuum on that for now. So the ROI is considered low.
Let's look at table three. A lot of data to vacuum. Even though the cost of vacuum is high, the storage savings are high. So the ROI is deemed medium based on the exact details of the vacuumable data. And predictive optimization goes ahead and schedules vacuum on tables one and three. It does not mean we ignore those other tables. We routinely check back on them to see if the ROI has changed and schedule vacuum on them if needed.
Okay, going back to how vacuum works. I simply said it, oh, it lists all the files for the table all the time. That's surely got to get expensive at some point, right? So, I tried this. I tried uh a table with 10 million files now and I ran vacuum. I ran vacuum on it. It took about 50 minutes and turns out 40 minutes a large chunk of that time was just spent listing these files. The first time you run the vacuum, you have enough data to delete.
This is probably okay. But if you saw if you see the third run of the vacuum over there, we barely had any data to delete and we still spent 40 minutes listing all the files. This can increase your compute cost rapidly. So can we do better? Introducing vacuum light. It's a new SQL command. No more listing files. Vacuum light leverages the delta log and identifies these unreferenced files from older snapshots of your data and reduces and removes the need for listing all these files.
So the vacuum uh I ran before which took 15 minutes now it it ran less than five minutes. It was awesome. And so the question here could be hey what about this other vacuum do we need to bother about that? So what we call full or traditional vacuum it's not needed to be run every time but it is still useful to clean up any partial rights on your table due to aborted transactions. It's definitely not required to run regularly if light vacuum is run regularly.
Now, when to run vacuum in which mode? Leave this to predtor optimization. With intelligent vacuum, predictive optimization figures out when to run full or light vacuum automatically. It keeps your storage cost in check and also efficiently uses compute resources. with predictor optimization enabled and intelligent vacuum can reduce your storage cost by half. Although vacuum light is specific to delta tables, the intelligent garbage collection and everything I mentioned here works for all managed tables including managed iceberg and declarative pipelines.
A quick recap. Automatic liquid clustering, automatic statistics, both are GA. And these two features combined and predictor optimization enabled can improve your query performance up to 20x. Don't just take my word for it. I'm going to pass the mic back to who's going to talk more about our adoption and how customers reacted to this. amazing. By the way, as managed tables can be read from external clients. Now, all these data layout optimizations that Naga has just presented also makes sure that your tables are indeed optimized even when consumed externally.

30:33 Adoption and customer success

So, manage tables are indeed both open and performant. And we're super humbled to see the overwhelming adoption and reception we see from customers. Since we have G8 predictive optimization, it has had tremendous adoption. We have more than 17,000 customers with predictive optimization enabled. We have more than 270 pabytes of data under management. We have compacted more than 60 pabytes of data, clustered more than 160 pabytes of data, and vacuumed over 600 pabytes of data, which is truly phenomenal.
And you don't have to just take our numbers for it. Predictive optimization has endorsed by many customers. Customers like Ankor have reported a 2x improvements in query performance and 50% storage cost savings. While some other customers really like how predictive optimization is an autonomous system and can fully replace manual solutions like the one at Toka AI, an AI data annotation platform. Many companies like Toka spend many hours identifying their most important tables, try to monitor their data and query patterns just to determine the best operations to run.
But as soon as they tried out predictive optimization, it worked so well for them that they decided to retire their own maintenance solutions right away. We also heard from the team at Planitude, a large energy company where they saw significant storage cost benefits soon after enabling predictive optimization. It's worth noting that all the optimizations that Naga has just presented, we only charge for the compute of the operations run, not the intelligence.
Making sure that we can truly help you lower your total cost of ownership with predictive optimization. And all this endorsement show that using manage tables and predictive optimization is truly an investment. But with all these AI powered capabilities, you might be wondering how do we keep track that predictive optimization is actually doing a good job. This is why we're making sign significant investment in observability to give you the best confidence as part of your data intelligence journey.

32:43 Observability and insights

First, let's see what might be some of the questions you might want to ask about predictive optimization. You might wondering what is the cost of running predictive optimization? What operations do we actually run? You know, what is the total estimated storage cost savings from turning it turning this on? What are our total bytes compacted, clustered, vacuumed, and analyzed? And of course, what are some of the top tables that were compacted, vacuum, clustered, and analyzed?
Luckily, we have a predictive optimization system table today that lets you track what optimizations predictive optimization has performed. This table logs all the operations that we have run on your tables. So you can simply query the system table to answer the questions you might have in mind. For example, you can use the example query on the slide to find which tables predict authorization has run the most compaction.
But the challenge here is that you have to create a query each time to answer these questions which in indeed require manual effort and it's it could be also difficult to set up explicit monitoring to compare like a before and after impact of predictive optimization. That being said, predictive optimization is all about making data management autonomous. So, can we do better? This is why we build a predictive optimization return on investment dashboard, which I'm excited to announce that will go beta very soon.
Let me walk you through the dashboard. For this example, all the usage data actually comes from our internal data platform. So, this is real usage data. So, as a user that first land on this dashboard, you can see the optimization coverage. And for this account, we're enabled on almost 99% of the tables. We also show you how much predictive optimization has saved you. Here you see that we only spent 30,000 overall on the feature, but it actually saved us almost a million dollars in annual storage cost savings.
Now, let's zoom into each operation. We can see how much has been vacuumed, compacted, clustered, and analyzed. and the top tables that has benefited from this. You may also notice that the top tables that we run these operations on are very different. This is precisely because of what Naga just shared earlier. The intelligence of predictive optimization lies in the fact that it only runs operations that gives you a positive ROI and each table has very different data and usage patterns.
That is why operations are on run on different tables at different cadence. And for once I can actually make sure that my tables are indeed well-maintained and optimized. You can also filter by catalog and schema or even date range if we have specific interest in narrowing down the insights. Last but not least, in addition to the dashboard, we're also working on an out ofthe-box system table to provide better insights into your manage tables, including the size and data layout health.
So stay tuned on this one. Now let me share about the road map ahead of us for predictive optimization. So looking forward we'll enable automatic liquid clustering by default on all UC manage tables. So cluster by auto as of today is still a knob you have to set on all your tables. So we want to elim eliminate that completely so you have the best data layout out of the box for the best price performance when using manage tables.

36:03 Roadmap and conclusion

And what does that mean? By simply running the create table command, you actually have autolid enabled by default for you. Moreover, we're excited to announce that we'll be launching a new capability automatic time to live by telling us the desired time to live for your data set with the set TTL command. Predictive optimization will automatically delete the data that has expired the days as specified in the time column.
This is super helpful for making sure that your data is compliant with legal and GDPR regulations and also improving your storage cost efficiencies. So stay tuned for our private preview in August. Last but not least, we're continuously improving our heristics and innovating with predictive optimization. We have more capabilities to improve how your data is laid out in your tables for the best price performance. This includes row roadmap it icons ahead of us such as automatic feature enablement on your tables, automatic archival of your data to a cheaper storage tier and automatic compression and more.
So just to just a quick recap, predictive optimization is super easy to use. It gives you a completely hands out hands-off data management while being highly observable. It can improve your query performance by up to 20 times and make sure that your system is much more efficient with intelligent vacuum. If there's only one thing we take away from this talk today is that use manage tables and predict optimization so that all your data management will be handled without a lift of a finger.
Last but not least, we're only halfway through the summit. If you're interested in learning more about liquid clustering, which is our innovative data layout technique that powers the automatic liquid clustering you heard about earlier today, please come to our liquid session tomorrow. If you're interested in our manage iceberg offering, which is also powered by predictive optimization, we have a talk on that as well tomorrow.
And if you're curious to learn about a real life example of using predictive optimization for cost management and how our own data platform team have saved hundreds of thousands of dollars on storage, go check out their talk later today. So it's super awesome to meet everyone today. If you have any questions and feedback on predictive optimization, please scan this QR code and fill in the short form. I'll follow on the responses and connect with you after the summit.
Thank you.

Learn more about the Databricks Data and AI platform.