Skip to main content

Introduction to Databricks Lakehouse

Summary

  • This video provides a lightning tour of the Databricks Data and AI platform's lakehouse architecture, explaining how it combines the flexible storage of a data lake with the reliability and speed of a data warehouse in a single open, unified, and powerful system that eliminates the silos and complexity of managing two separate tools.
  • Live demos cover creating a SQL Warehouse, exploring tables under Unity Catalog governance with permissions and lineage, building visualizations including bar charts and Sankey diagrams, configuring AI/BI dashboards, and setting up threshold alerts for automated notifications.
  • External BI tools such as Tableau Desktop connect directly to the lakehouse through built-in connectors, and the video shows that a fully functional SQL Warehouse can be created in approximately five minutes.

Introduction to Databricks Lakehouse

Watch: Introduction to Databricks Lakehouse
If you are brand new to Databricks Lakehouse and want to get a lightning tour of this intelligent data warehouse, this video is for you. Learn about the lakehouse architecture, then see how simple, streamlined interfaces are making it easier for analysts, developers, admins and business users to get their jobs done and questions answered. We’ll show how easy it is to create a warehouse, get data, transform it, build queries, create dashboards, and share with BI tools and applications. By the end of the session, you’ll be able to build a SQL warehouse in 5 minutes.
Talk By: Justin Kolpak, Sr. Solutions Architect, Databricks ; Kaitlin Baumgardner, Sr. Product Marketing Manager, Databricks ;

Chapters

FAQs

What is the Databricks Lakehouse and how does it differ from a data lake or data warehouse?

The Databricks Lakehouse combines the flexible, cost-effective storage of a data lake with the clean organization, reliability, and query speed of a data warehouse—all in one place. This eliminates the data silos and added complexity that come from managing two separate systems. The lakehouse is open with no proprietary lock-in, unified with built-in governance and AI capabilities, and powerful enough to support analytics, AI, and machine learning workloads on the same foundation.

How quickly can you create a SQL Warehouse in Databricks?

This video demonstrates that a SQL Warehouse can be created in approximately five minutes by providing a name, selecting a cluster size, and clicking create. Once provisioned, the warehouse is ready to explore tables, run queries, build visualizations, and connect to external BI tools. This fast setup is one of the core benefits of the serverless approach available in the Databricks Data and AI platform.

What is Unity Catalog and what does it provide in the lakehouse?

Unity Catalog is Databricks' unified governance layer that manages permissions, lineage tracking, and business semantics across all data assets in the lakehouse. It provides a single place to control who can access which data, see how data moves through pipelines, and apply consistent metadata and tags. This video shows how Unity Catalog integrates directly into the SQL Warehouse experience, making governance accessible to analysts and developers without extra tooling.

How can external BI tools like Tableau connect to the Databricks Lakehouse?

External BI tools such as Tableau Desktop connect directly to a Databricks SQL Warehouse using built-in connectors, allowing analysts to query and visualize lakehouse data without leaving their preferred tool. This video includes a live demo of Tableau Desktop integration with the lakehouse. Because the lakehouse is built on open formats, there is no proprietary lock-in and data remains accessible to the full ecosystem of BI and analytics tools.

Full transcript

[00:07] Hi everyone. Hope you're having a fantastic summit so far. Really appreciate you joining us for introduction to Data Bricks Lakehouse. I'm Caitlyn Bonggardner. I'm a senior product marketing manager for Lakehouse here at Data Bricks. >> And I'm Justin Coac. I'm a senior solutions architect here at Data Bricks.
[00:20] And with so many incredible new products and innovations rolling out this week, we wanted to take this section session to take a step back and really anchor on the core fundamentals of the data bricks lighthouse architecture. So I'm going to share some of the core features and
[00:32] benefits and then I'll hand it over to Justin to walk you through several live demos so you can see it in action. Our goal today is for at the end of this session that you are completely confident in spinning up your very first warehouse.
[00:45] All right. All right. Now, when you look at the full data bricks data and platform, as you saw this morning in the keynote, there is a lot happening here. Um, our platform does a lot covering everything from building smart AI agents and custom apps to agentic security and
[00:58] unified governance with Unity catalog. So, to orient us, we're focusing our attention right here at the bottom and that agentic data layer. So, this is that solid unified data foundation that
[01:10] is critical to making sure that those advanced apps and AI tools actually can do their job effectively. >>
[01:24] lake for storing large amounts of data cheaply and a data warehouse for actually organizing that data so analysts can run reports and get insights. This using two tools creates data silos, increases costs, and
[01:36] generally adds complexity to your environment. The lakehouse solves this by combining the best of both worlds. takes the flexible storage of a data lake and adds the clean organization, reliability, and speed of a data
[01:49] warehouse all in one place. And that's why we say the best data warehouse is a lakehouse. So, as a result, the lakehouse gives you three major benefits. First, it's open,
[02:01] built on completely open standards, no proprietary lockin. This means that you have total freedom to move and use your data um across a massive ecosystem of tools without worrying about it being trapped.
[02:13] Second, Lakehouse is unified. This is a fullfeatured enterprise data warehouse. So instead of buying separate systems for data cleaning, business reports and AI everything including your governance is built in. And finally, Lakehouse is
[02:27] powerful. We deliver robust everexpanding capabilities and significant performance gains year-over-year. In fact, just since last summit, out of the box performance has increased 42%.
[02:41] And you don't just have to take my word for it. We are consistently recognized by Gartner, MQ, and Forester wave reports as a leader. And more importantly, we're trusted by over 12,000 customers globally who are all scaling their operations on this open
[02:53] lakehouse architecture. So, let me walk you through a map of the journey that your data takes inside of Lakehouse. This first step, ingest. This is where we pull the data in from all of your sources. We call this raw data the
[03:07] bronze layer. In transform, we clean up the data. We normalize it. We get it in a predictable state. That's silver. And then the final layer, gold, is where your curated business level tables are that are ready for recording.
[03:21] Next, we run SQL queries to actually explore those tables and find answers and insights. And then we can easily easily visualize those answers on clean interactive dashboards. Finally, we can serve those insights out
[03:33] to external apps where the rest of the business can use them. All right, so let's start at the beginning spec specifically on spinning up a warehouse and ingesting data.
[03:45] The setup and configuration process for Lakehouse is designed to be as straightforward as possible. So all you have to do is give your warehouse a name, choose a cluster size, and click create. It's going to launch pretty much
[03:57] instantly and it handles heavy traffic automatically. So if you have a large team of analysts who are simultaneously running queries, uh the system automatically scales up to handle that. Uh it load balances and it manages
[04:10] workload cost intelligently as well. So you can set things like an inactivity timeout and the warehouse is going to shut down when it's not being used. So now that you have the warehouse set up, let's look at how we ingest data
[04:22] into it. This is where LakeFlow Connect comes in. The focus here is on making data loading as efficient and clean as possible. So, it's an end to-end incremental process. It doesn't just load your data once. It
[04:35] continuously and intelligently brings in new and updated data on an incremental basis, saving time and computing power. Second, it features built-in connectors. So whether your data lives in applications like Salesforce,
[04:48] traditional databases, cloud warehouses, uh the connectors are built right in. And finally, governance is unified. Your access controls, your data auditing, lineage tracking, all of that's built in
[05:00] from day one. So you know exactly where the data come came from and who can see it. And because of our open foundation, we work seamlessly with applications, databases, many different file types,
[05:13] uh, so that you can ingest from pretty much any source. All right, enough of me talking. I'm going to hand it over to Justin so you can see it in action. >> All righty. So, okay, I'm going to put my hat on. I'm a data analyst and let's
[05:26] just say that my data engineering team has gone through that work. They've ingested a data set and it's ready for me to use and they say, "Okay, now go into data bricks and just get started." So, we're going to talk through how we can do that using the data bicks lakehouse and how quick and easy it is
[05:39] to get started. So, what I'm going to do first is head over to my data bricks UI and navigate to the SQL tab. And then from there, I'm going to go and create a SQL warehouse. Now, to do that, I'll go under SQL warehouses. I already have a
[05:53] starter warehouse created for me here, but I'm going to show you how to create a new one. So, we will create SQL warehouse. I'm going to give it a name. We'll call it data summit 2026. And then from there, configuration is pretty straightforward. I give it a t-shirt
[06:06] size. I could set an autostop threshold so it spins down if we're not using it. And I can also set a scaling factor. So if we have a lot of concurrency, we can scale up to higher needs. And then when those needs kind of drop down, we don't
[06:18] need to spend as much money on those. So it's automatic and dynamic. Um, all with the configuration here. So I will go ahead and create a serverless warehouse. Now behind the scenes, data bicks is provisioning the infrastructure for me.
[06:30] Um, so I can start analyzing my data on the lakehouse. So just like that, we can see that our warehouse is running and we can start actually analyzing the data that has been ingested into our lakehouse. So what data set are we going to use today? Um, we're going to use the
[06:42] bake house data set. So this is one of my personal favorite data sets. It is a data set that comes built in with every databicks workspace. So if you navigate over to the catalog, we could see that there is this bakehouse schema that sits
[06:54] within the samples catalog. And again, every data bricks workspace has this. So feel free to play around with this, you know, after today. Um, and here we are in Unity catalog, which you've probably heard that term before, but this is the
[07:06] single pane of glass for governance across all of the assets on your lakehouse. So you can govern structured data, unstructured data, machine learning models, and much more, all through a single pane of glass. So we
[07:18] can see we have a couple tables here. Um, because it's a bakery, we have information on our customers and our franchises and suppliers. Also things like transactions. So I can take a look at the data set and get an understanding
[07:30] of you know what is the description of this table. What are the columns that are in the data set? There's a lot more but I'll quickly show the sample data. So this will actually use the warehouse that I set up to go run a query to just
[07:42] grab the top you know 25 or 100 rows from the data set. So we can see here we have transaction ID, customer ID. Um and now we have a better feel for the data set that we'll be using today. So with that you know enough exploring the data.
[07:54] I want to actually use SQL to analyze this data because like I said, I'm an analyst. SQL is my language of choice and I'll kind of show you how I can use the data platform to explore that data using SQL. So for that I'll head over to
[08:07] SQL and click on the SQL editor and I can do things like you know edit SQL. It is a SQL editor after all. So this is kind of my workbench for writing queries you know iterating on them. So
[08:19] in this case I'm running a select star. I'll make sure that I'm connected to a warehouse. So in this case, there's the data in AI Summit 2026 warehouse that I created, but there's also the starter one that was in there previously. So I
[08:31] can run queries. I can see the results below in a tabular format. And one thing that I find really powerful about the SQL editor is it's not just an editor. It's also a collaboration tool. So if I write, you know, the most sophisticated
[08:43] cool SQL query that's going to, you know, change the way that we look at the business and I want to share with my co-workers, I can go ahead and share that with my colleagues. So, for example, I can say, "Hey, Caitlyn, I want to share this with you." And I can
[08:55] go share this with her. And then you have a Google Doc like collaboration interface where we can both be editing the query in real time, taking a look at the results, and working together. So, I'll just run another query for the
[09:07] sake of conversation here. Here we're doing something slightly more complex, but we're just looking at customers by country. Um, and here we go. So, now you kind of get a feel for the SQL editor experience. We can write queries, edit them, see the results, and also share
[09:19] them with our co-workers. So keep this data set at the back of your mind. We're going to come back to the bake house later. And with that, I'll pass it back to Caitlyn. >> Awesome. So I'm going to share a little bit more
[09:31] about how we query and visualize and then we'll get back into that demo. So as you saw in Justin's demo, that SQL query editor is going to be your home base. Um the interface is made to is built to make writing and running
[09:43] queries as easy as possible. Three key features to highlight here. First is built-in AI assistance. Um, so the editor is automatically going to help fix syntax errors, explain complex code, and help you generate queries from
[09:55] scratch. As Justin mentioned, you can collaborate and edit in real time. Uh, so a lot of team members can jump in and all work on the same code together. And finally, you can store queries directly in Git. So this brings those engineering
[10:08] best practices in by letting you version control your scripts and connect them right to your team's workflows. But one of the main ways that you're actually going to probably interact with your data is with Genie, which you heard
[10:21] a lot about in this morning's keynote. Genie one is a great way to interact with your data in natural language, acting like a data smart AI co-worker. And then similarly, Genie code can help with those more complex SQL and Python
[10:34] queries, as well as doing things like build pipelines, fix bugs, and other advanced capabilities. It's also going to continually evolve to be more helpful the more that you use it. And again, you maintain that
[10:46] consistent control. It sits on top of the permissions, lineage, business semantics, and overall organizational guard rails that are defined by Unity catalog.
[10:59] Now that we have our queries, we also want to visualize the insights and answers that we get. And that's where data bricks AIBI comes in, helping you easily create clear and actionable dashboards. So, as the name suggests, it does have AI at its core. This makes it really
[11:12] effortless to create beautiful dashboards. Um, it also has Genie integrated. So, if your business users who are working on this dashboard have a follow-up question, they can ask Genie about that same data set. They can even build additional charts and graphs if
[11:25] they want to dig into a specific area. Second, it does deliver near instant performance because it sits directly on your SQL warehouse. Your charts are going to load and filter in seconds even if you're querying a huge amount of
[11:38] data. And finally, it's very secure because you're not moving your data out to a third-party tool that might have different business definitions or different access controls. You can really share it with confidence with your team and customers knowing that
[11:51] it's following all the governance that you've already set. And with that, let's actually see it in action. >> All right, cool. So, here we are again. We're playing
[12:03] with our big house data set, but like I showed before, that was just the raw version of the data. And we would typically call that the bronze version of the data. But like Caitlyn alluded to, we typically would clean that data, turn it into a silver, turn it into a gold version of the table where we're
[12:16] doing things like standardizing column names, conforming on data types, doing basic calculations, just things that will make the data set more consumable for end users, whether they're technical or non-technical. So in order to do
[12:28] that, I'm going ahead and transforming my data using SQL like we see here in this editor. So I've gone and created a bunch of tables which again will apply those rules doing things like casing and basic calculations and you know mapping
[12:41] certain values into the way that our business might used to see in them. So pretty basic ETL. Um I can define this all in SQL and I can go ahead and run this in the SQL editor like we saw before. But in reality I probably wouldn't do that. I'd probably want this
[12:54] to be automatic and something that's scheduled to run every night. So without leaving the SQL editor I can go ahead and actually schedule that. So, if I go over here to the top right, I can go to schedule and then this will set up a job
[13:06] using LakeFlow, which you heard about today during the keynote in the morning. Um, and let's say I want this to run at 5:00 a.m. Um, so every night at 5:00 a.m., these tables will run and recreate themselves. And then by the time I get to the office in the morning, have my
[13:19] first coffee, um, my data is refreshed and we're ready to go for the day's work of analysis. So, excellent. We have our data. It's been transformed. It's clean and ready for the business to analyze. Now let's kind of go through what would
[13:31] that workflow look like to analyze some of the structured data using data bricks lighthouse. So I have one example query here. It's pretty basic but you can imagine this can get much more complex as you know business needs arise. So in
[13:43] this case I want to look at a daily sales trend. So I'm writing against my transformed tables my gold layer tables which I'm calling you know fact sales and dim date. I can see the result set
[13:55] here in the bottom. So, I'm looking at the calendar date and looking at metrics like revenue, orders, units sold, and average order value. Now, I don't know about you all, but when I look at time series data in a table, it's kind of really hard to tell the trend line. Um,
[14:09] so for that, I want to build a quick visualization. I'm going to show you how you can do that without leaving the SQL editor. So, if I want to see this as a visual, I can click over here to add visualization and then pick my
[14:21] visualization type. Right here I'm going to do a bar chart, but you could see there is a wide variety of different visualizations you can create. So, you know, have fun with these. My personal favorite is a Sanki diagram, but the big house data set didn't have a good use
[14:33] case for it. Um, but there's a bunch of different visuals you can choose from. I'll go with bar for today. And I will put calendar on the X and then maybe revenue. And let's look at orders on the Y as well. And then I'm going to make
[14:47] them on different axes so I can see that a bit cleaner. Um, and very cool. Now I could see the data set a bit cleaner looking at revenue and orders. I can see the trend line. I can you know visualize the data without interrupting my
[14:59] workflow without going out to a separate tool. So I find that having this visualization capability built into the SQL editor is really powerful for those quick types of iterative analysis. Okay. So we've talked a bit about how we
[15:11] can analyze structured data on the lakehouse, but like I alluded to before, we also have very valuable unstructured data here within the big house data set. So, one of those forms of unstructured data is customer reviews. So, somebody
[15:24] comes into the bake house, they try a cookie or a croissant or whatever, and they leave us a review on Google reviews or Yelp or wherever they do that. Um, we're going and ingesting these reviews and we can see them here in a table
[15:37] format, but our marketing team thought, hey, wouldn't that be really cool if we could analyze the sentiment on these at scale so we can understand things like, are people liking our new products? Are people disliking them? What specific feedback do they have? and they want to
[15:50] be able to analyze that at scale. So that's where AI functions come into play. AI functions are a way to bring the power of AI to the comfort of your SQL editor. There are many different forms of AI functions that have
[16:02] different use cases, but the one we'll look at today is called AI classify. So AI classify works by providing an input. In this case, the input is the customer review. And then we're giving it different classes or categories that we
[16:14] want to classify the information into. In this case, I'm giving it three different categories, positive, negative, or neutral. And then just a quick description of what each of those labels means. And then I can go ahead and run the SQL.
[16:28] In this case, it's going to give me one column just in the raw JSON and then one column where I'm cleaning it up and to see what the label looks like. And then just like that, I have an AI powered classification model all without leaving the SQL editor. Um, and I think before
[16:42] AI functions existed and before this was possible, this would have required potentially weeks of work. I would need to set up Python environment, set up a Python environment, get LLM keys to an LLM provider, maybe work with
[16:55] my data science or MLOps team, um, and kind of have all these different tools stitched together in order to bring this AI to the data. Now, with AI functions, I can do this all without leaving the SQL editor, and folks that just know SQL
[17:07] can leverage that power for their analysis. So, I think this is a pretty cool capability. Hopefully, you guys can find some time to try it. And remember, AI classify or AI functions more broadly. Um, okay, cool. We're going to take a step away from the SQL editor and
[17:20] start talking about dashboards. So, SQL editor is great for when you want to write SQL or better yet, have Genie code write SQL for you. Um, but at the end of the day, it's not a full-fledged dashboard capability. If you want to
[17:32] build a dashboard that looks pretty and is interactive and you want to share it across your organization, that's where AIBI dashboards comes into play. So, we'll go over to our dashboards tab under the SQL tab. And then I already
[17:46] have one built today. So, we'll take a look at our bake house sales analytics dashboard. I'm going to go to edit draft here just so we could see what it looks like behind the scenes. And there are two different tabs when you consider a
[17:58] dashboard. And there will be more in-depth sessions later in the summit on dashboards. So definitely check those out if you want a deeper dive. But for the purpose of today's conversation on the data side, we are defining our data sets. So this is a place where you'll
[18:11] define the queries, how to extract the data from the tables in Unity catalog that you want to use for your visualizations. So we have our query. We also have our dimensions. So these are the things that we might want to group by or have visualizations include these
[18:24] dimensions. We also have our measures. So what are we actually calculating in this case? Nothing too crazy going on here. We have some sums, some counts. We have some compound measures that we'll use to measures to define ratios and
[18:36] things like that. Um but for example, total revenue is defined as the sum of the total price field. So over here on the data tab, that's where you're defining all that information that you want to use in the context of your
[18:48] dashboard. And then over here on the right, this is the canvas view where you can create one or multiple tabs depending on how you want to organize your reports that actually show the visualizations. So one trick that I like to do is I
[19:01] usually never start a dashboard fresh. I'll always have GD code take a crack at it. So just to show what that experience looks like, if I go over to create new dashboard, it'll prompt me with this text box where I can just say, "Hey, I
[19:15] want to use this bakehouse data set. these are the types of measures I want and these are the types of visualizations. And this will bootstrap a pretty impressive dashboard and then from there you can iterate on it, make sure it looks and feels the way you want, add new visualizations and build
[19:28] from there. Um, and that's actually exactly what I did for this visualization, but we'll kind of show what the final product looks like. So for each of these visualizations, it's going to look very similar to the in SQL editor view we saw before in that you're
[19:40] defining a visualization type. You're also defining what goes on the X, what goes on the Y, and then there's additional configurations to make it look and feel the way that you want it to look and feel. Um, so we'll take a look at a few more. Looking at revenue
[19:53] by payment method, revenue by franchise. Um, great. So, this dashboard looks awesome. I'm ready to go share this with my organization. Feeling good about it. To do that, I'm going to go over to publish.
[20:06] And then I can publish this dashboard and make it available to folks in my organization. Um, and again, I might want to share this with Caitlyn. Um, she can share it with her team and we can make this more broadly accessible. But if I want to make this available to her,
[20:18] all I need to do is share this, click add, and then we're good to go. Um, all of the data underneath is still governed by Unity Catalog. All the permissions are going to be respected. Um, and all without needing to stitch together
[20:30] multiple tools. All the data is centralized here on the platform, all governed with Unity Catalog. Okay. So, I love dashboards. I personally don't think dashboards are going anywhere. Um, but it turns out a
[20:42] lot of folks don't really like dashboards because they're reactive. And what I mean, what I mean by that is you need to look at a dashboard in order to understand the trends and see what's happening in the data. And a lot of folks want to have things that are more proactive by nature. So instead of
[20:56] needing to log in, why don't you tell me when something meaningful changed in the data? So that's where alerts come in. I will show you an alert that I created here for sales trend. So when you define an alert, you're doing two things. The
[21:09] first thing you're doing is you're writing a query. Again, query written in SQL. You could write this in your SQL editor. Um, but it's all using the same data that's governed by Unity Catalog. So I'm writing my query that defines the metrics or the data that I want to be
[21:22] alerting on. In this case, I'm calculating a week-over-week sales percentage change. And then the second thing I'm doing is looking at a condition that I want to set based on the result of this query. So here I'm
[21:34] saying if the percentage change field is less than negative -20 or another way if sales have dropped more than 20% week overw week I want to be notified about that. I don't need to have to log into a dashboard. I want you to tell me in my
[21:46] email or in Slack or Teams or wherever you live and work um I want to be proactively notified when that happens. So I write my query. I set my condition and in this case I've set myself as an email recipient to this notification.
[22:00] So, we'll take a look to see. Unfortunately, sales were poor. They they went below the 20% threshold. And then I get an email that looks something like this. I was nervous. I was going to show my whole inbox. But glad you all
[22:13] can just see this one email. Um, great. So, sales trend alert was triggered. Um, from there I can see that, you know, it dropped 37.86%. Now, from here, I can go view the alert.
[22:25] I can go look at my dashboards. I can start chatting with Genie to understand what is the trend in this data and more importantly what could we do about it. So I've just showed a lot but to help digest here we we've shown how we can
[22:37] use data bicks lakehouse to analyze structured data unstructured data using AI classify and other AI functions and also built full-fledged dashboards and proactive alerts using the data bricks alerts feature. So I'll pause there pass
[22:50] to Caitlyn to bring us home. All right. So, you could just stop here. You have clean, organized data. You have beautiful dashboards. They proactively alert you whenever you need it. Um, but
[23:03] for many teams, there are still going to be some external apps that benefit from the data as well. And that's where this final step, serve, comes in. Lakehouse connects to a very broad ecosystem of business intelligence tools
[23:16] that a lot of teams use today. Excel, Google Sheets, PowerBI, Tableau uh can all easily pull from Lakehouse. There are also a number of connectors, APIs, AI agents, and apps that we can serve to
[23:28] as well. So, I won't go into a lot of detail about this here, but if you're interested in learning more, there is a session uh tomorrow afternoon on how to connect Excel and Google Sheets specifically. And then on Thursday, there's a broader session about BI and
[23:41] partner tools that you can connect to. So, if you're interested, highly encourage you to attend that. All right, let's look at how it works. >> Cool. Me again. Hey everyone. Um, hi.
[23:54] Okay, so the reality is not every team lives and works in data bricks. You know, people are using other BI tools like Tableau, which we'll take a look at in a second. People are building custom applications, but they need the data that's in the
[24:07] lakehouse, and we need to be able to serve them in a seamless manner. So, we'll talk through how we can do that. I will go to my over here on the SQL warehouse tab and we'll take a look at our data and AI summit warehouse that we
[24:19] created earlier. I'm going to start it just for fun. Um so how do we connect to it externally? If we go over to the connection details tab over here at the bottom, this is going to give us all the information that we need to connect to
[24:31] this warehouse um through, you know, JDBC, through more generic means like a REST API, but this is all the stuff that you would need to put into a connection string to connect to this warehouse. For our most common and popular connectors,
[24:44] we actually have built-in packages to make that really seamless. Um, so we'll take a look at how that works with Tableau as an example. So let's say I want to connect to datab brick SQL through Tableau. I can go ahead and click on this and then download a
[24:58] connection file. This is something I can open up from Tableau desktop and it's going to come pre-built in with all the configurations I need to connect to this warehouse. Then when I'm building reports in Tableau and refreshing things, it's actually going to fetch the
[25:10] data from the SQL warehouse behind the scenes. So I'm not going to download this now, but I've already have an example ready to go. So we'll open up Tableau Desktop and then this is what that connection file looks like. It's going to come built in with
[25:24] our connection string information. So I can see here this is the information to connect to the host, the information to connect to the warehouse specifically. And then this particular example uses OOTH. So I can connect through my data bricks login. Um but that's all I need.
[25:38] So I do that. I can then pull in the specific tables that I need. I can model the data. like in this case I'm bringing in fact sales and saying how it's related to dim customer. Um but this is all the normal data prep that I would do
[25:50] if I were using Tableau as my BI tool. Then from there I can go build visualizations that do whatever I want. So in this case I have a bar chart that's looking at you know total price revenue um by country. So then from
[26:02] there I can publish this to the Tableau online or Tableau server um and then behind the scenes whenever we need to get fresh data it'll go to the SQL warehouse that we configured earlier. Okay, so like I said, dashboards are
[26:14] awesome. They're not going anywhere. But I think we've all seen the rise of custom applications with things like vibe coding. Every organization is building dozens, if not hundreds of custom applications that are
[26:26] purpose-built around their business processes, which is really cool and exciting, but we still have the same problem of we need to be able to get data from the data bricks platform into those applications in a seamless manner. So how can we do that? Well, if we go
[26:40] back to our connection details tab, you'll see here we have a bunch of connectors. There's actually more than the ones that are listed here. Um, but I'm building my application in Python. So, we'll walk through that example of I want to build a custom application that
[26:52] does more than just a dashboard, more interactive, more AI integrated. Um, but again, I want to get data from the SQL warehouse. So, I will click on Python. And then this will give me all the instructions that I need to download the
[27:05] packages necessary. And then um here's the boilerplate code I would need to go and start writing queries against the SQL warehouse using this Python SDK. So I will go to the app that I have built. In this case, this app is
[27:19] deployed as a databicks app, but it doesn't need to be a data bricks app. This could be an app that's deployed anywhere um just for the sake of conversation. So we can see you know there's basic things like dashboards. It's just using like a custom formatting
[27:31] that fits within the brand for the bake house company. um but it's also doing more extended functionality. So in this case, this is something you couldn't necessarily do in an application or like a dashboard, but you would do in a
[27:43] custom application. So we have this revenue scenario simulator where I can, you know, apply these different inputs and then I can see how, you know, for example, our cost forecast might change under different conditions. So the
[27:56] details of this aren't as important, but the main point is you can build custom applications that extend the functionality of dashboards while they're still pulling data from that SQL warehouse under the hood. All with the governance that Unity catalog provides.
[28:10] And then from there, you can bolt on whatever you want, things like an AI analyst, things like anomaly detection algorithms, but the key point is behind the scenes, it's all governed. It's all streamlined through the SQL warehouse. >> So I'll pause there. Back to Caitlyn.
[28:23] >> All right. So today we covered a lot of the basics of the lakehouse. Um but as Ali said this morning, we've released over a hundred new features just since this time last year. Um features cover many
[28:35] different data types, different ways to manage and optimize your performance, AI functionality, ecosystem integrations, migration support. What you see on this slide is actually just what we worked on in the second quarter of this year. So,
[28:49] we're always expanding capabilities, new features, uh, new performance improvements. Really, Lakehouse is going to always be ready to handle whatever your team needs.
[29:03] As we start to wrap up, if you want to learn more here at Summit, a few s uh sessions I would suggest that you attend. First is a deep dive. This happens tomorrow afternoon. Uh, it's a full 90 minutes, so we do go into a lot of detail about a much broader range of features for Lakehouse. So if you want
[29:16] to go in kind of the the 202 to this 101, that's the session I would recommend. Next, comprehensive data warehouse migrations with Lakebridge. LakeBridge is our free tool for moving your data to the lakehouse. So if you're
[29:29] not on Lake House today, but after this session you really want to be, Lake Bridge is the best way to do that. And then finally, as Justin mentioned, if you're interested in learning more about dashboards, uh there are two sessions Wednesday and Thursday afternoon on
[29:42] dashboard creation and intelligent insights. Last ask from us, please do complete those surveys that you'll find in your app. We really appreciate your feedback. That's how we make Summit better
[29:55] year-over-year. Uh so really appreciate you doing that. And with that, I think we have some time for questions.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.