The Modern Databricks Notebook: Tips, AI workflows, and Best Practices for Every Practitioner

Summary

This video covers productivity features in Databricks Notebooks that most practitioners never discover, including side-by-side editing, tab groups, result table profiling, filter-to-SQL conversion, and an inline catalog explorer.
Genie Code, the agentic AI partner embedded in Databricks, can run parallel tasks from the side panel, create and edit cells across the notebook, refactor entire notebook structures, and assist with cross-notebook analysis workflows.
Engineering best practices covered include the notebook debugger, unit testing, serverless session restore, variable rename, and cross-notebook refactoring — all illustrated through a live end-to-end Bake House analytics scenario.

The Modern Databricks Notebook: Tips, AI workflows, and Best Practices for Every Practitioner

Databricks Notebooks are the heartbeat of data and AI work, but most practitioners only scratch the surface of what they can do. Whether you are writing your first cell or building production-grade pipelines, this video is a practical guide to getting more out of the modern notebook experience.
Topics include:

Core authoring productivity: side-by-side editing, tab groups, TOC and minimap, and smarter cell management
Running and exploring data with result table profiling, filter-to-SQL, and targeted SQL execution
AI in your notebook with AI functions for SQL and Python, plus Genie Code for agentic development
Engineering best practices with the notebook debugger, unit testing, and Serverless session restore
A live end-to-end workflow that brings everything together

Talk By: Samantha Banchik, Product Manager, Databricks ;

Chapters

FAQs

What is Genie Code in Databricks Notebooks?

Genie Code is an agentic AI partner embedded in the Databricks workspace that lives alongside you in a side panel within notebooks. It can create and edit cells, refactor entire notebook structures, run parallel tasks, and perform cross-notebook analysis — acting as an intelligent coding collaborator rather than just a code completion tool.

What hidden productivity features does Databricks Notebooks offer?

Databricks Notebooks offer side-by-side editing, tab groups, a table of contents, a minimap, and a keyboard command palette for navigation. Additional capabilities include result table profiling, filter-to-SQL conversion, and an inline catalog explorer that lets you browse and preview data without leaving the notebook.

How does serverless session restore work in Databricks Notebooks?

Serverless session restore allows users to resume a notebook session with Spark state preserved, avoiding the need to re-run cells from scratch after a session ends. This video demonstrates cloning job history into a notebook while keeping the Spark state intact, which is highlighted as a best practice for production-grade workflows.

How can Genie Code improve notebook performance?

Genie Code can analyze notebook code for performance issues and suggest or automatically apply improvements. This video shows Genie Code performing performance improvement checks, refactoring the code structure, and resolving bottlenecks without requiring the developer to manually trace the root cause.

Full transcript

[00:08] Hi everyone. Uh my name is Samantha. I'm a product manager here at Databricks, and I'm super excited to be talking with you all today about Databricks notebooks. This is actually historically been one of our most popular talks here at Databricks. So, you're amongst the privileged few to get to experience this
[00:20] live. Um I've been at Databricks for the last 2 and 1/2 years or so, and I've had the privilege of talking to a ton of different notebook users, uh some of which may even be in this room. And one persistent pattern has constantly
[00:32] emerged where there is a vast difference between the types of features users are actively taking advantage of in notebooks compared to all the different features we actually have to offer within notebooks. So, my goal for today's talk is simple. I want every
[00:45] single person in the room today to learn at least one new feature they didn't know existed in notebooks before. And ideally after this talk, the next time you're authoring a Databricks notebook, I want you to actively use that feature. So, with that being said, let's go ahead
[00:58] and get started. So, I'm going to assume everyone in the room today already knows what a Databricks notebook is. They've been the front door for data work at Databricks since the dawn of time. It's where you do all of your data exploration, data
[01:11] engineering. You might be building data sets for machine learning, and it's where you have maybe historically also collaborated with your peers. And none of that is changing. That's not what we're here to talk about today. But what has changed is what notebooks
[01:23] fundamentally represent as an asset. So, notebooks are still going to be your source of truth for all data work. It's not only going to be where you investigate, build, and share data, but it's actually going to be where you and
[01:35] your agent investigate, build, and share data. In this case, the agent being Genie code. If you don't know what Genie code is yet, uh we've probably thrown it at you quite a bit throughout this conference. But just as a friendly reminder, uh Genie code is your agentic
[01:48] partner that lives alongside you within the Databricks workspace. Um so, whereas historically when authoring in notebooks, you might have been more so focused on actually getting really nit-gritty and writing the code. You might have been using Genie code or
[02:00] what was formerly the assistant to diagnose code errors, but now the way in which you operate and interact with notebooks is actively changing. Instead of doing all of your core work within the notebook itself, you're now doing all of your core work within Genie code.
[02:13] And in that way, the notebook isn't just a representative asset of all of your data work, it's the representative asset of your agentic reasoning, agentic investigations, and your collaboration with Genie code as your partner. Uh with that being said, I could talk about this
[02:26] for ages, but I think what would actually be better is to go ahead and show you a live demo of all of these different agentic features in practice, and also potentially go over some notebook features you might not have known existed before that could be
[02:38] potentially really valuable uh in making you more productive in the way you work. So, with that being said, I'm going to switch over to a demo.
[02:58] Cool. So, before I get started, awesome. So, for the purposes of today's demo, we're going to do a little bit of role-play. Uh let's say I am not in fact a product manager here at Databricks, but instead am a member of the data team
[03:11] at what is going to be called the Bake House. Uh the Bake House is the hottest uh coffee and pastry chain. We started off in San Francisco, and we've done so well, we've now internationally franchised. And part of my jobs as a member of the data team was building a
[03:24] notebook that got translated into a job that sort of summarizes all of these international insights uh for all of our different Bake House franchises on a weekly basis. Um so, let's say going into the week, I know my manager has a really important
[03:36] board meeting at the end of the week. Uh she's likely going to want me to build off of this existing job. All the while, I might have some colleagues who might have some questions that will involve iterating off of this job. Uh I know it's a little bit strange to be starting a notebooks demo from the job page, but
[03:49] I promise after this we'll spend the entire rest of this demo uh in notebooks. Um but we can see here that historically this job takes about 30 minutes to run. And if I want to iterate on top of this and take advantage of the Python and
[04:01] Spark variables that might have been created during this job, in order to do so I'd have to navigate to that notebook and I'm sure nobody here wants to sit here and watch this notebook run for 30 minutes and 44 seconds. So, we've actually introduced a new feature as a
[04:14] part of this clone into notebook process. We're now instead of simply uh opening up the historic job and having it run in order to iterate on top of it, I can actually clone the notebook and also clone its historic state. So, I'm
[04:26] going to clone this notebook. And in just a few seconds, we can see here that all of the run state from that historic job run has also been included in this notebook. It's also okay to clap
[04:38] if you're excited. Awesome. And if you don't believe me, I can actually prove this to you as well. Um if I go into the variable explorer here, you can see here that all of my historic uh Python and Spark variables
[04:51] that I had for my last job run are now readily available here. You can only imagine how useful this would be if you have like a long-running job that results in an error. You can now debug it instantly without needing to rerun your entire notebook again. So, really
[05:03] awesome feature. We're super excited to have launched. Um so, before I kind of dive into any asks my manager might have before the end of the week or I might dive into any asks my my colleagues might have, I think what's really important to know is
[05:15] sort of the underlying data that's powering this notebook. Um and so, in order to do that, notebooks have this really awesome embedded catalog explorer feature. That's nothing new. But what I think is uh particularly special about it is that it actually highlights assets
[05:28] that we think are particularly relevant to you, whether they're the uh tables that are using actively within your notebook or ones that you've actually recently referenced within the catalog explorer. Um and so, instead of actually opening up it up uh using this button
[05:40] directly, I'm going to use this as an opportunity to show another really awesome notebook feature uh that we launched last year called the command palette. So, I'm going to hit command shift P and we can see here we actually have a really awesome overview of all of the
[05:53] different keyboard shortcuts and all of the different tasks that I can do within notebooks, making a really seamless experience for me. Instead of me having to sort of sift through the UI and all of these different buttons in order to figure out how to do what, I can simply have all of that readily available here
[06:06] and I can search over it as well. Um and we can see here actually at the top it has exactly what I've been looking to do, which is open up the catalog explorer. As you can imagine, I've been running through this demo a few different
[06:19] click that and the catalog explorer is uh readily available for me. And like I mentioned, it's not just a great way for you to view all of the tables within UC, but it's also a really awesome way for you to get an overview of all of the tables
[06:31] actively being used within your notebook. Um so, in just a moment you'll be able to see when this table was last updated, who updated the notebook, um also popularity information, so really valuable things and I can even take this
[06:44] table and preview it within uh the notebook itself. And I can also even expand this, learn a little bit more about all of the underlying columns. Again, none of this is necessarily new, but it offers me a really awesome way to get a handle on which tables are actively being used in
[06:57] this notebook, so I know um what I can sort of author on top of moving forward. Another really awesome feature is our table of contents. So, let's say I know that my manager's meeting at the end of this week is a board meeting
[07:09] specifically with the US board. They're going to want to dig into the US franchises for the Big House specifically. Um luckily for me, when I originally authored this notebook, I had a lot of handy-dandy markdown cells that offer me a really awesome way to better
[07:22] navigate my notebook. Um so, like I mentioned, my boss is particularly interested in franchises. This is a relatively long notebook. I don't want to scroll around to try and figure out um the franchise section that I'm searching for. So, I can actually go
[07:34] ahead and click here and with the click of a button, I've scrolled through six different cells, no extra work on my hands. Um I'm simply exactly where I want to be to sort of learn more about the franchises.
[07:46] So, I don't know about everyone else's boss here, but mine is all about the numbers, and I know that they're going to want to have a really great understanding of revenue across the franchises going into their board meeting um this Friday. And so, looking
[07:58] at uh sort of this total revenue column here, I can see here that I did a really awesome job sorting it, but it doesn't really give me a good idea of the distribution of these values. I could go ahead and write some SQL or Spark code to sort of get that better
[08:10] understanding, but notebooks actually have this really awesome feature directly embedded within them where I can get this overview of this column without having to write any code myself. And you'll notice throughout this demo, I'm not writing any code myself. Um, so
[08:23] I can actually go ahead and click on this column, and then click on open selection details. And we'll see here we have this really awesome uh distribution chart sort of giving me a really great overview of
[08:35] this column. I can see here the minimum value is 177, whereas the median's around 1,200. There are probably some franchises that are underperforming. I can even scroll over uh this really nice histogram that's created for me and see
[08:47] that uh we have about probably three different franchises that are sort of falling behind compared to the other ones. So, just a really awesome way to get an overview of this column without me having to write any code myself. Additionally, yes.
[08:59] Amazing. Um, additionally, you can also use this to compare two columns at once. So, if I wanted to let's say compare transaction accounts or transaction counts with total revenue, I can select both of those. And we can see here selection
[09:12] details actually expands and gives me an overview of both of them. So, also offers a really awesome way if maybe you're um attempting some different translations across multiple columns to make sure those sort of persist um and look as expected. So,
[09:25] awesome. With that being said, I'm now going to pivot over to some really exciting AI features we have embedded within the product. Um, so there are actually a number of different ways you can interact with Genie code today within notebooks. For the purposes of this
[09:39] demo, I'm going to primarily focus on using uh Genie code in the side panel. You can actually open this in one of two different ways. Uh this button up here at the top and also this button over here at the side. Awesome, and we have a new thread. So,
[09:52] let's say my Slack has been popping off during this demo. My manager is asking me for a customer breakdown across US-based franchises. I have another colleague that wants me to take a deeper look into revenue attribution and flag
[10:05] which franchises are underperforming. So, for the next step of this demo, I'm going to show you how Genie code can not only do these tasks for me, create new notebooks in order to do this investigation, but I can actually do both of these tasks for me in parallel.
[10:17] So, um to start, I'm actually going to copy some pre-made prompts I have over here. So, I'm going to tackle my boss's ask first because that's most important. I'm going to say create a new notebook called uh Big House Customer Analysis,
[10:30] and I'm going to tell it where to put it. I'm going to say in and I'm actually going to at-mention the folder that I specifically want Genie code to put this analysis a new notebook in. Um you can see here, this is actually a relatively new feature within Genie code. You can
[10:42] at-mention a bunch of different types of assets, really allowing you to have sort of a more curated, specified response within Genie code and have some assurance that it's working with the exact assets you want it to. Uh you can imagine how valuable this is if you
[10:54] have, let's say, multiple different tables that might have similar names, the agent could potentially get confused. Uh I will say here it does actually a pretty good job with vague prompts, but for the purposes of this demo, I'm going to simply tell it to put it in this folder. I'm going to give it
[11:07] an additional instruction to use the revenue definition that's defined within this notebook in the next notebook carrying forward, just to also prove that it can take in context from assets outside of the one it's creating. So, I'm going to say use the same
[11:19] revenue definition that's in this notebook. And finally, I'm going to give it the task that maybe my manager pinged me on Slack, which is going to be to show the
[11:32] top US customers by total spend segmented by state. And I'm going to go ahead and send it off. While it's doing that, I'm going to open up another Genie code thread. So, this is also a relatively new feature where now you can have multiple Genie code
[11:45] threads within the side panel and have multiple tasks running at once. I'm going to spin off the revenue attribution task that my colleague gave me. So, again, I'm going to go back to this prompt to markdown file and start it in similarly the same way.
[11:58] We're going to say create this folder and put it in again the same folder that we had before. Sorry, create this notebook in this folder. Similarly, I'm going to say use the same definition for revenue that's in this
[12:10] notebook. Again, I'm being intentionally vague here, but I could have at mentioned this notebook directly. And then I'm going to give it a little bit more detail on what exactly I want, which is to
[12:22] break down revenue by franchise and flag any underperforming franchises. And I gave it sort of a definition for how I want to do that. And I'm going to go ahead and spin that off. And now we can go ahead and check in on
[12:34] the progress of the last one. We can see here we actually have a blue dot, which indicates there's a task that needs my attention and some sort of human in the loop in order for the agentic to continue. So, I can swap over there. We can see here it wants to run the SQL. This is
[12:46] going to be really annoying. I don't want to have to constantly go back and approve every single tool call. I know I can adjust the settings here. We've actually recently introduced this new setting for auto approval, which I'll go ahead and open in the settings panel and set it. Where now, instead of me
[13:00] having to allow every single tool call within this agentic flow, I can simply let it run. And ideally, at the end of this, Genie code will really neatly have a notebook created for me without me necessarily needing to switch back and forth and constantly be approving
[13:12] things. And I don't necessarily need to always be in the loop. So, with that being said, I'm now going to go back to the next one, which actually worked a lot faster than the first one, which is awesome. And we can actually navigate to the notebook that it created directly. So,
[13:24] I'm I'm to go ahead and click this asset here. We can see here it's I mentioned it a couple of different times in the prompt. You can actually click on any of these and it'll open it up as its own soft tab. I'm going to go ahead and click this one.
[13:37] And we can see here all of the generated Genie code code that it gave me. For the purposes of this demo, I'm not going to really neatly go in and check on all of these and accept them on an individual cell level, although I
[13:49] could do that. I'm just going to accept all of these as is and run it and make sure everything runs successfully. So, I'm going to go ahead and click accept all here. There are a couple of different ways places you can accept everything, but I'm just going to use the side. I'm going to go ahead and run
[14:02] all the cells in this notebook. I'm going to actually use this as an opportunity to show off another relatively new notebooks feature, which is this cell mini map we have over here. Similarly to the table of contents, it
[14:14] offers you a really awesome way to sort of jump through any sort of notebook without any scrolling on your behalf. Additionally, it also shows you the execution status of all of the code cells within your notebook. So, you can actively get a high level see if any
[14:26] cells failed, where within the notebook they failed, and instantly jump to failing cells. And we can see here that both of these ran as expected. Genie code did exactly what I asked. Yay. No one's clapping.
[14:39] And I can actually, like I mentioned, go back and check on the next thread. We can see here by this green check mark that the thread is actually completed. So, I'm going to go ahead and click that tab. And I'm also going to go ahead and open up this one.
[14:54] And I'm again, just for the purposes of this demo, going to accept everything. And run it. So, for those of you who might recall when I was originally authoring the prompts, I gave Genie code a very explicit instruction that I wanted to
[15:06] ensure that the revenue definition was taken from the original notebook and persisted across these other notebooks. So, there are a couple of different ways I can actually either use Genie code to validate that this definition is consistent, or I can actually leverage
[15:18] the notebooks UI directly to ensure that this definition is consistent. So, I'm actually going to go ahead and close this for now. And I'm actually going to drag this notebook over here. So, I get this really awesome sort of
[15:30] side-by-side view of um both of these notebooks that have been generated. We can see here that sum of total price was used uh for revenue and it actually really clearly defined that for me at the top. And similarly, in this other
[15:42] notebook we had Genie code generate for me, it also used sum of total price. So, it was able to really clearly follow my instruction and bring in that context from the original notebook into the two new notebooks I created.
[15:54] So, I'm now going to go ahead and move this back as its own soft tab. So, so far I've showed you how Genie code can sort of leverage context from one notebook and take it into a notebook. I've shown you how Genie code can create net new notebooks from
[16:07] scratch, but Genie code can also within a single prompt make edits across multiple notebooks at once. So, let's say for the purposes of this demo, I want to add a signature to the two new notebooks that we've created um and maybe also the original source notebook
[16:20] for the job. I'm just doing this for demo purposes so that you can see that Genie code can uh create new cells across multiple notebooks at once. So, let's say I want to say create a markdown signature cell in all the notebooks in
[16:34] the demo folder. And I'm going to say have it say hi from Samantha at Dice. And we'll see here in just a moment how Genie code, again, it's not just great
[16:46] at authoring singular assets at once or bringing in context from other assets into a single asset, but can actually generate recommendations across multiple notebooks within a single prompt. Uh and this is a really simplistic example, but
[16:58] you can imagine how useful this would be when it comes to authoring across notebooks with multiple dependent or authoring across jobs with multiple dependent notebooks or creating pipelines. Um and so, like I mentioned, in just a moment it should be creating these markdown cells across all these
[17:11] different notebooks. Awesome. And if we wanted to review all of these, we've a really awesome way to do so. We can actually click in here and validate that Genie code did exactly what I asked for. Again, I can do this
[17:23] across all of the different notebooks. Just to see the markdown recommendations. But again, really awesome way to sort of leverage Genie code to not just author single assets and single cells, but multiple ones.
[17:36] So I'm going to go ahead and accept everything. Let's say as a next step in my process here, I'm really proud of the regional performance notebook I put together and I want to share it with my colleague, but I actually think there's really
[17:48] valuable insight in the thread that I used in order to create this regional performance notebook. So we've actually recently introduced a new feature where you can not only share the notebooks that you created with your agent, but you can also share the threads that you had with Genie code that you leverage in
[18:01] order to create your agent. So I can actually simply click this three-dot menu here, click the share button. And I don't know if other folks in the room actually knew this, but all threads are actually notebooks under the hood. So this screen should look fairly
[18:14] familiar to you. You can share them in the exact same way you could share a notebook today. So again, just a really awesome way for you to sort of share what you and the agent did together with your colleagues.
[18:28] Awesome. So let's go back to the customer analysis notebook that I made for my manager, who again has a really important board meeting at the end of the week. And my manager I know for a fact is really interested in visualizations. And we can
[18:40] see here that Genie code, at least in its initial pass at giving me this analysis, hasn't generated any for me. I know my manager is also really particular in the way that the visualizations work look. He wants a particular look and style and feel and
[18:52] color scheme. So I'm actually going to take a screenshot that I took from a previous presentation, give that to Genie code and have it generate an image that emulates what is seen in the screenshot. So I'm going to go ahead and click this plus button. Note all of
[19:05] these different logos for now. I'll get back to those in just a moment, but I'm actually going to click on the attach files button here.
[19:18] And go ahead and pull up this reference image. And this is actually going to be really, really small, but I'll zoom in just in case folks can see. We can see here there's a lot of specifics that my manager might want uh this visualization to entail. It should have a dark background, bar should be in turquoise.
[19:31] We can see uh sort of in the same Databricks color schema, a red to indicate underperformers. And so, I'm going to again going to go back and reference my prompts here.
[19:45] And go back here. And I'm going to tell Genie code to use the style in this image as a reference, same color palette, same formatting approach to generate a customer spend chart in this notebook to match the exact style that we see here. And
[19:57] importantly, I'm also going to tell Genie code to remember to use the style in the future. And we'll get to in a moment why exactly this particular part of the prompt was so crucial. So, I'm going to go ahead and send Genie code off.
[20:10] And while this is working, I'm now going to go back to this plus button and talk about uh some of these enterprise connectors you'll see here. This is actually a feature that we have in public preview today across our entire Genie product suite. MCP servers in
[20:23] Genie code are not relatively new, but we found that they were a bit tedious to set up over time. So, to make that easier for the most popular MCP servers, we now have enterprise connectors. Admins can enable them on an admin level, and then simply through OAuth,
[20:35] users can connect to any and all that they might potentially be using. So, you can imagine in this particular scenario, I actually took a screenshot, I gave it to Genie code, but let's say the screenshot came from a Google slide somewhere. I could have actually
[20:47] authenticated uh using the Google Drive connector here, just simply pasted the link to the slides, and have Genie code sort of proactively bring in that context for me, uh ensuring that I don't actually have to screenshot and uh pull things in myself. So.
[21:00] Uh we can see here it's going ahead and editing uh this notebook. We can use the cell mini map here to figure out which cell specifically is being edited. It looks like it just ran. And awesome. We can see here that Genie code was able
[21:13] to do exactly what I instructed. It created a bar chart for me that really similarly matches the style of the image that I uploaded. I can go ahead and share this with my boss, no problem. So, I'm going to go ahead and accept that code.
[21:25] And now going to go back to what Genie code is actually doing now, which is sort of why remember to use the style in the future was so important. Um so, let's just say this isn't the last chart I'm probably going to create using Genie
[21:37] code in Databricks. I'm I don't necessarily want to have to re-upload an image every single time I want it to follow this particular standard or give it maybe a link to a Google slide. So, Genie code actually has proactive
[21:49] memory where it sort of learns as you continue to use it and actually gets better as you continue to use it. So, because I told Genie code to remember to use the style in the future, it actually proactively added it to my instructions file, which I seamlessly go ahead and
[22:01] open and see here that it was able to remember my chart style preference specifically. And all of this was pulled out from that image. I didn't have to do any sort of additional prompting myself.
[22:13] So, I'm going to go ahead and accept it. We didn't go over Genie code customizations and personalizations today. So, Genie code also has skills embedded directly within it. So, right now having added this as an
[22:25] instructive, this is proactive context that Genie code will consider regardless of what my prompt is. So, as you can imagine, I'm not always going to be asking about generating charts. Throughout this demo, I also had it generate notebooks for me. And so, this
[22:37] context might always might not always be applicable. So, I can actually further furthermore after this have Genie code translate this into a skill, which is essentially a proactive piece of context that's agentically pulled in when applicable. So, now in the future if I
[22:50] were to translate this to a skill, it would only pull in this context if it knew I was creating a visualization or a chart. So, with that being said, that's going to conclude the live portion of this demo. I'm now going to switch back to slides and walk through a couple of new
[23:03] notebook features that I didn't get the opportunity to demo today, but that I think will be particularly exciting. So, I'll switch back.
[23:23] Awesome. So, like I mentioned, I would have loved to demo all of these different features today, but unfortunately not all of them are ready. Um so, any and all the things that I show you here will either be available shortly, either post data and I summit or within the up and coming months. Um so, the first thing that I'm
[23:35] particularly excited to show you all today is Genie code comments. Uh so, we all saw as a part of the demo today that you can actually proactively share threads with your colleagues just like you could share any other notebook, but maybe you want those conversations to
[23:47] live within the place they are actually authoring, next to the cells you're working with. Um and so, now just in the same way that you can at mention a coworker in a comment, you can now also at mention Genie code, effectively starting your own mini threads directly
[24:00] within the notebook. In this way in the future, when you go ahead and share notebooks with your coworkers, you'll also be proactively sharing the comments within them, those comments now also including conversations you've had with our agent.
[24:14] This next one I'm also really, really excited about. Um and this slide actually demonstrates two different things. Um so, the first is that we're actually revamping our version history within notebooks. So, now we'll be delineating the difference between uh code changes you make yourself versus
[24:26] code changes that Genie code makes on your behalf. And we'll not only be allowing you to sort of view two different uh notebook runs side by side and the different code changes within those notebook runs. Uh it's very, very faint on this slide here, uh but you'll
[24:39] notice here that this filter actually caused three new rows to be created. Um and so, we'll also proactively be showing you, again, not only the code changes, but how uh different notebook runs have persisted changes across your data and your actual cell results. So,
[24:52] really awesome way for you to sort of understand how your data is changing over time. Yes, very exciting. Um we're also going to be shortly offering a presentation mode, which is
[25:04] just another way to view notebooks. Um I mentioned that I'd be sharing sort of my analysis with my boss at the end of the week. Uh maybe my boss isn't necessarily interested in the underlying code. They trust that either myself or Genie code
[25:16] on my behalf of myself has correctly generated this code, um and they only necessarily care about the outputs and the visualization. So, this will be sort of a way in which you can share notebooks as a presentation asset rather than like sort of a code-based asset.
[25:32] This next one is also particularly exciting. Um we're now going to start offering multi-file find and replace. This will essentially allow you to search across every single file within your project, preview all of the changes as a diff before you actually commit to replacing them. Um if you've ever had to
[25:45] manually hunt down a variable rename across a dozen different notebooks, this feature is for you. Of course, as I demonstrated, you could use Genie code to do this for you today, but this is just another opportunity to do that that may work even faster potentially.
[26:02] This next one I'm also really, really excited about. So, we are actually going to be bringing a local IDE-like project authoring experience directly to notebooks, so they are now no longer isolated in their own compute and environment. Uh in this way, uh I'll
[26:14] kind of walk you through all of the different numbers you might see here. Um the first is that the Git folder space becomes sort of a shared working directory for both humans and agents. Um number two is that we are now sharing a serverless compute across all of your
[26:26] different notebooks within a project, so no more having to manage different serverless sessions across all of your different assets. Um you'll also be able to have a file as sort of your source of truth for all of your dependencies across your entire project, and the web
[26:39] terminal will no longer be isolated by specific asset. It'll be shared amongst all of your assets within uh sort of your Git folder. So, a really awesome way to sort of manage multi-file projects within Databricks. I'm I'm excited about this. I see some nods in
[26:52] the room, so I think others are as well. Uh finally, this one uh is going to be particularly beneficial for me as someone who writes a lot of different SQL in notebooks. Um for those of you
[27:04] who don't use SQL in notebooks often, uh what will happen every single time is we automatically create this variable {{}underscore} SQL DF on every single SQL output. And for me, what will often happen is I'll run this at the top of
[27:16] the notebook. I'll get to another SQL cell at the bottom. I'll realize I actually want to reference the previous output from another SQL cell, and then I'll have to rerun it to um rename this variable or maybe have Genie code um end up translating it into a temp table. So,
[27:28] we're actually now going to be offering the ability to rename this with Genie code or to rename uh the variable definitively by actually double-clicking here and being able to rename it uh proactively yourself, so saving you from those potential uh variable name
[27:41] collisions in the future. Finally, this is another feature that I'm really really excited about that has uh shortly sort of been rolled out at least partially. We're now will be flagging which queries within your
[27:53] notebooks are actually potentials for optimization. Um and we will actually give you proactive recommendations with Genie code. So, upon every single SQL query within your notebooks, if you look at the bottom right-hand side, you'll
[28:05] see whether or not Genie code checked for improvements, if it flagged any potential uh performance improvements you could be making, um or identified things like task skew, missing filtering keys, or exploding joins. Um we will actually flag it to you as a user, and
[28:17] you'll be able to simply click on a button, and Genie code will give you a really awesome performance optimization recommendation actually rooted in uh your query performance insights, so giving you even more intelligent recommendations on how to make your
[28:29] notebooks faster, more performant. Awesome. With that being said, that is going to conclude um this entire notebooks talk. Just as sort of a recap over everything we went over, notebooks
[28:43] aren't necessarily just the place for all of your data way work and collaboration with your colleagues. They're now also the place and asset where you collaborate with your agent. Um we also went over a handful of not necessarily AI features, but common notebook features that I think a lot of
[28:56] practitioners miss and could potentially take advantage of. Things like the cell execution mini map, the table of contents, result table profiling, um and finally we went over a bunch of awesome new features that may or may not have
[29:08] been shipped as of today, but you should all get really really excited about. Things like query performance insights, multi-file find and replace, uh shared compute across your projects. So, like I mentioned at the beginning of this talk, I hope you learned at least one new
[29:20] notebook feature you didn't know existed before. And I really hope you take that new notebook feature with you the next time you work in Databricks. So, uh final note is that any and all feedback is always welcome. Um we have a
[29:33] ton of these different feedback links embedded throughout our notebooks product in the Databricks product in general. I promise you an actual human person goes ahead and reads every single piece of feedback you submit. So, please go ahead, whether it's positive or
[29:45] negative, reach out. Um we are constantly trying to make notebooks better. We know it's the primary way in which people interface the Databricks. Super super important to us. Additionally, uh Erin, our designer, who I think is in the back, yeah, she's
[29:57] waving, uh will be doing a bunch of different brain dates throughout the rest of the conference and will also I think also be at the developer relations booth if I'm not mistaken. So, please feel free to find her. She'll be happy to show you some prototypes and some of the things we're working on and further give us some feedback. So,
[30:11] with that being said, I think I ended about 5 minutes early. So, I'm happy to open up the floor to questions. Otherwise, I hope everyone has a great rest of Data and AI Summit. Um and hopefully you have fun at the train spotters tonight. So,

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.