Sponsored by: Microsoft | Unlocking the Microsoft Data & AI Ecosystem with Azure Databricks

Summary

Azure Databricks is a native first-party Microsoft product that launches 28 million VMs per day, integrating with the full Microsoft ecosystem including Microsoft Entra ID, Power BI, Microsoft Fabric, Copilot Studio, and Agent 365.
Recent announcements include Power Platform general availability, automatic identity management, SAP BDC integration, LakeFlow Connect, and OneLake catalog federation for zero-copy access across Fabric and Databricks.
Genie is now available for both business users and data scientists, with new integrations bringing it directly into Microsoft Teams and Copilot Co-work environments for in-context analytics.

Chapters

FAQs

What makes Azure Databricks different from Databricks on other clouds?

Azure Databricks is a native first-party Microsoft product, not a marketplace solution, meaning it is co-engineered with Microsoft and deeply integrated with Azure services such as Microsoft Entra ID, Azure Data Lake Storage, and enterprise security controls. As noted in this video, the service launches 28 million VMs per day, reflecting the depth and scale of the Microsoft–Databricks partnership.

What is OneLake catalog federation and what does it enable?

OneLake catalog federation is a generally available feature that provides zero-copy access across Microsoft Fabric and Databricks, so teams working in either platform can see and query each other's data without moving it. This eliminates silos between organizations that use both Fabric and Databricks and reduces the need for duplicate pipelines.

How is Genie being integrated into the Microsoft ecosystem?

Genie is being brought directly into Microsoft Teams and Copilot Co-work environments, allowing business users to ask natural language data questions without leaving their collaboration tools. This video describes this as part of a broader strategy to meet business users where they already work, reducing the friction of switching to a separate analytics interface.

What recent Azure Databricks announcements were highlighted in this video?

This video covers Power Platform general availability, automatic identity management, SAP Business Data Cloud (BDC) integration, LakeFlow Connect, and OneLake catalog federation general availability. A 21.1% performance advantage for Azure Databricks over other clouds and customer stories from the Cincinnati Reds, AT&T, Barilla, and CRISP are also featured.

Full transcript

[00:09] Well, hello everyone. Um, thank you for joining today. I'm so glad to see so many of you here. Uh, and personally, it's extremely I'm extremely excited uh to be back at the data bricks data and summit. We've done a lot in the last
[00:22] year. I honestly cannot keep up and I don't even think I'm going to get through all of this content. So, I hope you're rooting for me to get through all of this content. Um, and so if you could keep questions to the to the end because
[00:34] of that, um, I' I'd greatly appreciate it. My name is Annavina. I lead the Azure data lake storage as well as the Azure data bricks product group. I always say here at Microsoft because I'm
[00:46] usually at Microsoft doing doing these, but you get the message. Um, overall, I'm thrilled to be representing Microsoft at DIS. Uh it's an amazing opportunity and I'm so glad you're all
[00:58] here. Um remember to definitely take the surveys not just for this session but all of our sessions. Uh feedback really makes this super valuable. However, if you do not like the session, my name is
[01:10] David Meyer and I work at data bricks. Okay, I don't have a thing to scroll here, so I'll just be going back and
[01:22] forth. Um, but many of you are already aware that Microsoft and Data Bricks came together back in 2017 to really launch this strategic partnership. But it was not just a partnership. It was
[01:35] also this product called Azure Data Bricks, bringing the best of data analytics and now AI as well. It's truly innovation at enterprise level scale. If
[01:47] you had to guess, and you don't have to, I'm not going to put this group on a spot, but the Azure data brick service launches 28 bill million VMs a day. That kind of scale is really a testament to
[02:00] the usage, the battle tested Azure platform coupled with that enterprise security and scale that I'll talk about as well. So, it's truly a battle tested solution and our customers love it. Uh
[02:12] it's been a decade almost a decade uh and we'll continue this partnership for many more decades to come as well. And you're probably wondering hey what is this Azure data bick summit at the data bick summit and well maybe not if you're
[02:26] already here but you'll know you're using Azure data bricks when you see that familiar top left corner I've circled it for ease for you but that's when you know you're running Azure data bricks and not data bricks on other cloud providers. What that means is that
[02:40] it is a native firstparty solution. It is not a marketplace solution like it is on other clouds. And so we co-engineer, co-develop. I do sometimes feel like I
[02:52] talk to more data bricks team members than Microsoft team members in my day-to-day. But I think that is by design. We're really fully assimilated and make sure that these integrations that we build and that I'll talk to you about are truly successful.
[03:06] And hey, I'm not the only one saying that. A third party, Principal Technologies or PT, ran a performance benchmark and found that Azure data bricks was 21 no 21.1 that one is also
[03:19] important% faster than other clouds as a whole. And what makes this better on Azure in my opinion is how it's built into the Microsoft ecosystem. So giving
[03:31] you more speed, agility, and value right out of the gate. You don't have to build up teams to do the integrations, understand the technologies, patch, test, validate, and so forth. We take
[03:44] care of it for you. Let the platform do the work. You can just get the insights, analytics, and now all of the context with your agents and AI that you're building as well. You have a platform that's really going to scale with you no
[03:57] matter how your business grows. And of course, we have unified billing and a common support system. So you're not dealing with different bills from different providers or different support tickets. We'll take care of that for
[04:10] you. We'll work with data bricks if it's a data brick specific issue or if it's a Microsoft issue, we'll take care of that. And so that makes adopting or leveraging this this platform extremely easy for Azure customers, but really any
[04:23] customer in general. Okay, I alluded to this a little bit, but it's deeply Azure data bricks is deeply integrated into the Microsoft ecosystem. It's a firstparty and maybe
[04:37] that's a Microsoft Kool-Aid term, but it's basically just like any other Azure product. And so you'll hear me say this firstparty native term a lot and and that's what I mean when I'm saying that. And so we continue to enable seamless
[04:49] experiences in some cases even updating them and making them better. And I'll talk about that shortly as well. Let's take on this example uh on this circle or the ecosystem that I've drawn here uh
[05:01] event hub for example. So you can use the cafka compatible endpoint provided by Microsoft event hub as well as the structured streaming cafka uh connector from Azure data bricks and just use them
[05:14] together and you're good to go. So you don't need any thirdparty libraries. You don't need to figure out what to do. You're just connecting them and that's available in data bricks runtime as well. And so since there's no need for a
[05:26] thirdparty library, this works seamless with LakeFlow Spark declarative pipelines as well. Okay, what's next? Next is Azure data lake storage, right? And so we actually took a bunch of
[05:38] feedback from data bicks in the early days. This is the storage solution really built for battle tested big data analytics solutions making that truly successful. So when you're using Azure data bricks, all of your data is stored
[05:50] in your storage account and that's just ADLS as we call it. We've also invested heavily in directly publishing to PowerBI. And so in my opinion, which might be a little biased, it's the best
[06:03] BI solution out there, right? And so whether your teams and business users want to use um dashboards or get the insights on a weekly basis, monthly basis, and nowadays even on a daily basis, they're able to do that with our
[06:15] complete built-in integration where you can direct a publish to PowerBI. Of course, um Azure Data Bricks Genie uses Azure OpenAI models under the covers, but not only that, you can also
[06:28] use Azure OpenAI models or really any model that you want, assuming there's no government restrictions on them. um in Azure data bricks as well. So whatever model you want is available for you to use for your uh workloads.
[06:41] Continuing along the circle, there's also data factory. Um and so we have a direct connector with Azure data factory as well as fabric data factory. And so that way you don't have to have any
[06:53] custom work that you have to do off the get-go as well. The Azure datab bricks connector orchestrates data workflows by triggering uh data bricks notebooks, python scripts or whatever it is that you like. The key here is the context
[07:06] remains. So if you are in Azure data factory or fabric data factory, you don't have to leave and go to data bricks or Azure data bricks at all. Right? So keeping that context and so bringing it to where you work, where your users work, where your organization
[07:18] is as a whole. Of course, Azure Data Bricks is also available um as a Azure Data Bricks Genie actually is available as a um tool within Foundry tools
[07:30] catalog and that's something we just launched uh last year as well. And finally to complete my circle of ecosystem that I have here. Um we also have a connector enabling Azure data bricks to be a knowledge source for
[07:43] Copilot Studio. So maybe you're using Copilot Studio to build your agents. Great. get all of that rich context that Ellie was talking about using uh Genie right there in Copilot Studio. So really
[07:56] we've done a lot. There's also many more Microsoft products than I have and can fit on this slide. But the key message is this. You don't have to build the integrations because it's an Azure firstparty service. We build them for
[08:08] you. We maintain them for you. taking away time that you would have to sit and manage this and instead building you know the insights the innovation and the analytics. So okay let's take that ecosystem and and put it all together in a more user
[08:22] centric way. We have that Azure infrastructure at the bottom that compute storage and networking that battle tested scalable infrastructure and of course we're leveraging entry ID
[08:34] for authentication. We also launched automatic uh identity management for entry which I'll talk about as well. And so that's really the base leveraging Azure data bricks. And so all of the capabilities um within data bricks are
[08:48] available on Azure data bricks day one. There's there's really no difference. Whatever AI model you want maybe team A prefers anthropic and team two prefers Azure OpenAI. Clearly team two is the
[09:00] better team but hey who am I to judge? Um and then of course no matter how you come into the ecosystem right whether you're using Microsoft 365 my favorite teams uh or if you're using the power
[09:12] platform which is power automate um copilot studio as well as power apps or github um and VS code it doesn't matter how you enter the ecosystem you can still use the context the intelligence
[09:25] and the business data that you have through Azure data bricks of course you can also mirror data that's within Azure data fabrics and EDLS into Microsoft one lake as a whole but that's also not
[09:37] needed if you directly want to go to fabric uh powerbi you can do that too as the push to publish that I was talking about earlier so that's on the data side of the house let's switch over to the um AI side and so right from copilot studio
[09:51] up until agent 365 in fact when agent 365 was launched azure data bricks was a partner at launch as a whole and so if you're not familiar with agent 365 it's effectively Every agent has their own
[10:03] entry ID identification and then when they work and do kind of work it's you have that identity you're able to manage that across the board and so when you have agents that you build within Azure data bricks or let's say Genie for
[10:15] example they they can now be managed and completely traceable through agent 365 as well and so really it feels like you're working with knowledge a knowledgeable teammate who has always
[10:27] access to the current most up-to-date data as a So that's kind of the ecosystem and how it's tightly integrated uh together. And so I'm not just saying this because hey this is great but it's because customers
[10:40] are actually using this too, right? So this data intelligence that we apply to data warehousing delivers worldclass price performance without requiring any data movement. So you don't have to move your data. You can use your data where
[10:53] it is. There's no different system. There's no different syntax. In fact, you heard how it doesn't matter whether it's Delta or iceberg anymore. it's just going to work in Azure data bricks. Um, and then that enables you to have that continuous AIdriven optima uh
[11:06] optimization enabling your agents to get better as they get that more uh context. And all of this isn't hypothetical, right? This isn't something that is coming isn't something that is made up. It's something that exists today and
[11:19] we're already seeing global adoption across thousands and thousands of customers um and enterprises as well. every major industries here uh financial services, manufacturing, retail,
[11:31] healthcare, energy, you name it because they are getting that strong price performance that continuous AIdriven optimization that I was telling you about as well as this seamless
[11:43] integration with the Microsoft ecosystem. Now, this slide, I've decided that I'm no longer going to be adding more customers to it. It can only look good with this amount. I don't have Alaska Airlines in there. I don't have Inter
[11:55] Mountain Health in there. I don't have Emirates in there. There's so many customers that I just don't have in there. So, you're just going to have to believe me. In general, we always kind of I guess last year uh we showed this
[12:07] slide, but this year I wanted to do something different. I wanted to take four customers and go into a little more detail about how they've been successful on Azure Data Bricks. And it's not just AI, right? AI can be a part of it, but I
[12:20] want to go back to the basics a little bit because unless you have that foundation, all of the and the context well-built that's scalable, whatever agent you build on top of that is not going to be effective. Right? So, I'm
[12:32] going to share four customer stories with you and I'm going to ask you at the end which one was your favorite. Okay, let's start with the Cincinnati Reds. So, the Cincinnati Reds are one of
[12:44] Major League Baseball or MLB's oldest and most storied franchises. They operate in a very competitive environment of professional sports. So, marginal improvements in performance, preparation, and timing and strategy
[12:58] determine the outcome of the game. And over the past decade, the Red saw an explosion in the amount of data they have available to them. things like technologies such as IoT sensors, player
[13:10] tracking, fan engagement. There's so much data around sports these days that the volume and the complexity started to grow by as much as 100x, which is crazy if you take a moment to think about it.
[13:22] But their legacy infrastructure, their legacy data infrastructure just couldn't keep up. They were processing or trying to process billions of rows of data that took hours or even days. Can you just
[13:34] imagine sitting there waiting for data to process? Sounds horrible. And so as a result, it was nearly impossible for coaches and players to make real-time decisions to that then impact the game.
[13:46] So imagine you're trying to make a decision. The game is going, your data is still processing, you're nothing is happening. And so what is the point of having all of this rich data across all of the sensors that you have, but you're not able to make any decisions that are
[13:59] going to benefit your team or benefit your coaches? and people are instead spending time and energy maintaining the platform instead of using these insights in in in the real world for their use
[14:11] case as a whole. And so they finally were like, hey, if we want to win more games, we have to shorten the loop between data collection as well as decision-m, right? So the
[14:24] time that was taking those hours and days were just not cutting it as a whole. And so what they ended up doing is transforming their data infrastructure by using shocker to no one Azure data bricks right and so now
[14:37] they had automated data pipelines so that replaced their manual processes their infrastructure management was eliminated so t so team members weren't just managing infrastructure and wasting time that could otherwise be spent in
[14:50] getting the insights updating strategy updating player uh positions timings and things like that. And so this really saved them thousands of compute hours. And again, that's imp incredibly
[15:03] impressive. They saw a three to five times improvement in latency and an 83% reduction in pipeline um run times as a whole. And not only that, they also saw an reduction in cost in those compute
[15:16] hours that I was telling you earlier about. And so what they realize is the volume or the benefit of data is not in its volume. it's how quickly can I take the data that I do have, access it, get
[15:28] those insights, and then have turn that insight into impact um via action, right? And so that entire cycle time was just cut short and they could just immediately update. And so coaches had
[15:40] exactly the information that they needed at the right time. And none of this was with AI, right? I think we hear a lot about AI in this agentic world that we're in, but we got to build the foundation for that AI to be successful,
[15:53] right? Right. And this was just the foundation that they built. Um, and I guess go Reds. Okay, switching gears a little bit. Let's talk about AT&T. Anyone here use AT&T? Is their network provider?
[16:06] Awesome. Okay. Um, so AT&T is one of the largest telecom providers serving hundreds and millions of customers operating at massive scale, massive
[16:19] complex networks. Uh, and you can just imagine the amount sheer amount of data that they generate every second. In fact, the seconds that it took for me to describe what AT&T does, they probably had more data accumulate um, as a whole,
[16:32] like millions of transactions per second. Now, as you can imagine, AT&T relies heavily on data to deliver things like better connectivity for the people that raise their hands, um, but also
[16:45] preventing things like fraud, right? Can you think about how painful it is where a fraudulent activity has already happened and then you're being contacted that hey this might happen you're like okay dude it already happened like you need that to be proactive and not
[16:58] reactive and that's exactly what they were seeing right technical teams were spending incredibly insane amount of time to do these operational tasks like patching their data because it was coming in from so many sources it was
[17:11] not streamlined it was not unified and so they couldn't really benefit from these advanced use cases using AI as a whole. And so they recognized that, hey, in order to solve this infrastructure platform, we got to
[17:25] first of all move away from our on-remise infrastructure. It was costing them time, energy, and effort. And they also wanted to be able to do this all in real time. So what did they do? They unified their data analytics and AI
[17:37] platform onto a single platform. Any guesses for which one? Um, and as a result, they were able to accelerate that innovation. And so in terms of impact and so they took that insight that they had and in terms of impact
[17:49] they saw a 300% improve in ROI over five years. So think about that you had the data across multiple systems but you weren't able to access it and it was costing you time. So with Azure data
[18:01] bricks all of that went away and they saved time, energy and effort. They saw 3x faster cycles from recovering the uh recovering the data, understanding it and then getting that to action as a
[18:14] whole. The other thing is they saw 80% reduction in fraud attacks. So what they ended up doing is they actually built their own I think like 100 plus ML models um helping them for their
[18:26] specific use case to then prevent fraud for a customer. So if you're seeing lesser fraud calls on everyone who put their hands up that's because of the power of Azure data bricks as a whole. They were also able to shut down their um onremise
[18:39] data centers and that's also saved a bunch of cost to them. Okay, that was customer number two. Let's switch gears a little bit to customer number three, which is Burila.
[18:51] Anybody have Burla pasta? I personally love Burla pasta, especially the protein one. You got to get your protein in. They're one of the world's leading food manufacturers. Globally known for pasta, sauces, um, as well as bakery products.
[19:05] And as you can imagine, they're operating a very largecale manufacturing facility with complex supply chain. And of course, they have to rely heavily on this operational data all the way from production, manufacturing, logistics,
[19:18] and end of line as well. And not just that, this is food. So quality and consistency also matters, right? They have to do their quality checks on time. They have to have data about, hey, were these done? Which batches are affected?
[19:31] Which batches are not? because ultimately that comes down to customer safety as well. So think about all of this data that they had, but it was disconnected. So they couldn't really connect their production data to like their supply chain information or their
[19:44] customer engagement systems as a whole. So they had to work across these distributed systems and they had this limited ability to connect all of this data together. And so their processes were not not optimized for this real
[19:58] time information. And in this industry, there's a lot of changes, right? Like maybe you need to have less inventory because of something that happened at a specific store. You don't want inventory just lying in not in stores but in your
[20:11] own um manufacturing uh facilities as a whole. So they as a result what was happening was their decision- making similar to the reds was being delayed. And so what they did, any guesses on
[20:23] what they did? Seriously, they switched over to Azure data bricks, right? And so the power um for them was really that they could have a unified data platform across their
[20:35] manufacturing, production and supply chain giving them those insights as a whole. And that saved time of also on right so now their engineers were not spending times trying to connect these different
[20:48] units but using Azure data brick works as a whole and so really those real time manufacturing insights were now possible with Azure data bricks. I think these three case studies showed
[21:00] hey these are organizations and you know according to our mission we want to enable organizations to do more but for me my most inspiring is when people are using Azure data bricks to better the lives of the community so let's talk
[21:14] about crisp or the chesapeake regional information system for our patients as they call it this is a stated health information exchange that aggregates data patient data across provider ers
[21:28] public health agency and research institutions like John Hopkins to enable better care for people like you and me. And so um this is actually currently in the state of Maryland. And so since
[21:41] 2011, every hospital in Maryland uh has used Crisp as their messaging service as well as uh what they use for care. Since they had great success when they were leveraging Azure data bricks, they
[21:54] also expanded Crisp to something called CSS which is a customer shared service. Right? So Crisp shared services or CSS. Now the problem that they had is they wanted to combine hundreds of millions
[22:08] of dispar data sources and put them all together. So think labs, images, prescription, immunization into a unified view. They had all of the data but they couldn't put it at an individual patient level. as a whole.
[22:21] Their existing infrastructure had relied on API based aggregation query time which took too much time. So by the time you're visiting the doctor across these 12 states, they didn't have the information on your prior immunization,
[22:34] your health records to give you better care as a user. And before this, they would spend 12-hour shifts in order to just get the data that they needed. when they
[22:46] embraced Azure data bricks literally went to minutes a file is sent over and they're all set but what's the coolest thing about this is they started embracing this just during co and so in
[22:58] three weeks they spun up their entire Azure data data bricks infrastructure they were able to get questions on demographic data which were the vulnerable u populations where they should put diagnostic and testing events
[23:12] and so that to me is super cool because they were then able to direct uh resources to how the pan or how COVID was spreading within certain communities that were more vulnerable than ever. And so it's super cool that this is not only
[23:25] um a service that's you know of of course enabling companies uh to do better but also enabling the community and and you and I in and people like you and I in all of these states as well.
[23:37] Okay. Now probably what you all are most hear about. We learned about how across sports, telecommunication, healthcare, and retail, how customers are using
[23:49] Azure data bricks, how it's benefiting them, how it's taking time instead of them doing a lot of management of the infrastructure. They're just focusing on insights, analytics, AI, and innovation.
[24:01] Let's switch gears to recent announcements. I'm going to start with ones that we announced since last data and summit, you know, all through September and then chronologically come to um what we announced at build two weeks ago and then what we're also
[24:14] announcing today. So that's kind of the flow. Some of it you might have seen before, but I'll go deeper into some of them. So these ones are all generally available. Um I'll talk through all of
[24:26] them as we go. So the first one is Azure data bricks power platform connector. So as I talked about earlier, power uh platform consists of power apps, power automate as well as uh copilot studio.
[24:38] So because we have this connector which I think we actually announced uh in public preview at the last data NEI summit and then in September we announced uh GA a few months after that you can leverage this to connect to your power apps build applications that
[24:51] directly read and write to Azure data bricks right so from your power apps effectively you don't have to leave uh as a whole using power automate which I extensively use to automate a lot of
[25:03] workflows you can build flows and actions that execute using custom SQL um or an existing job and get back the insights that you want all without leaving p the power automate flow. You
[25:15] can also build custom agents using copilot studio and they can get all of the knowledge from the context within Azure data bricks as a whole and this is available today. You don't have to do any work to set it up. It's just
[25:27] literally clicking through it. So that's the first one. The second one is automatic identity management. So previously when uh we announced this in public preview and then a few months
[25:39] after in G and and now it's really the default uh it basically means that you don't have to do any manual user or group setups right so identity management is all taken care for you and
[25:51] so um even if you want to share a dashboard that you've created in Azure data bricks with someone who has an enter ID you're able to do that and it's all managed because of that integration um that we have as I mentioned earlier this is available in all new workspaces
[26:05] in a simple toggle. You can enable it in existing workspaces as well. The next one that we announced um last fall actually was the SAP BDC connector to Azure data bricks. So think about
[26:18] this, you have all of this rich data and context uh in SAP's business uh data cloud, but you're not able to cover all of that data when you're asking questions within Azure data bricks or building those insights as a whole. So
[26:31] with this connector, all you got to do is you have an existing Azure data bricks instance. You can literally just delta share uh from SAPBDC and you have that full 360deree view of your business and you're ready to go. There's no data
[26:44] movement. There's no cost as a result of that to move the data as a whole. And you enable this bidirectional live delta sharing all from the get-go. This was a tremendous work across SAP, Microsoft
[26:56] and data bricks. And it's super cool to see the usage and make customers lives um a lot easier. And then coming to FabCon in March of this year, uh we launched the LakeFlow
[27:08] Connect free tier first on Azure um followed by you know I guess the other clouds. Um but effectively LakeFlow Connect, you're probably familiar with this. Basically just a pointandclick UI
[27:20] to get that data into Azure data bricks as a whole, right? And so with the free tier, you now have about 100 um DBUs that you can use um for specific connectors just for free on the go. So
[27:33] that was super cool. We're seeing a bunch of customers already use it. It's like 2,000 plus at this point and it's great to see. And finally on at the bottom of the slide is the Azure data bricks mirrored catalog item. So this is
[27:46] within some context of our uh Microsoft fabric ecosystem. All mirroring is a concept is just reading right so think of me I'm here and if there's a mirror in front of me I can see kind of and let's say I'm the data I can see my data
[27:59] there but I'm not actually moving right so there's no data copy or zero data movement as we call it but you can access your Azure datab bricks insight knowledge analytics from one lake in
[28:13] Microsoft fabric right and so you're creating this um mirror as I was describing earlier so we announced that actually just in March in public preview and and just a few months later it's uh it's it's already G. So there's those
[28:26] are some five uh announcements that I wanted to highlight that are already generally available. Um this is another cool one which is the Azure data bricks addin for Excel. So let's say hey Anvie that's great but I don't access my data
[28:40] through I don't know copilot studio or teams or whatever it is. you're like, "Hey, I'm a hardcore Excel person." And apparently there's also like games where people evaluate who's better at Excel.
[28:52] So it's it's really like a cult following. I I love it. And so um this was one of our public preview announcements that we recently made. So effectively, your Azure data bricks Excel addin enables you to access and
[29:04] analyze your Unity catalog govern data all within Excel. So you're no longer making like a manual copy and working on that. is this live trusted data that you're working on even outside of Excel.
[29:17] And so when that governance is enforced through Unity catalog and Azure data bricks, you get that built-in consistency, security, and that single source of truth. And that's really the key. We don't want to work out of we don't want to move data. We want zero
[29:30] copy. We want it live and we don't want to duplicate, right? And so that's really um instead of making Excel like a siloed tool separate from Azure data bricks, it's now completely seamless and giving you that an analytics front end
[29:44] um for those Excel people to truly accelerate accelerate decision-m there's two new things on this that are also upcoming. One is scheduled refresh. Um an example for example, we get like our
[29:57] revenue data on you know week one of the following month for example. So I know that's going to happen. So I can schedule my refresh for maybe the following day so I get my revenue data up to date. Um at least that's how I um
[30:11] sometimes use it. And there's also the Genie integration. I feel like that's the topic um of this whole conference really. Um so you can then get your AI insights directly in Excel through Genie. So that's coming. Stay tuned for
[30:25] that as well. Speaking of Genie, uh I'm sure most of you are no strangers to this. Um this is really the Azure databicks AI experience for users right it gives you a single
[30:37] place within Azure datab bricks to ask data questions analytics in your natural language. Uh of course now Genie is new and improved as Ali announced yesterday with Genie 1 Genie spaces Genie code uh
[30:52] catered to different users across the board. I wanted to set some context about Genie and then I'll jump into announcements that we excuse me have as well. So just I'm a visual person so I just wanted to visual I wanted to help
[31:04] visualize where Genie sits right. So you have whatever format you're using. Of course, everything is governed with Unity catalog. So only people who have access to the data can access it. Um
[31:16] whether that's in our lakehouse architecture or you're using LakeFlow to get the data in or even Lake Base as a whole, all of that context is sitting within Genie. And so whether you're
[31:28] using Asian bricks, AIBI tools, custom Asians, it has all of the context from Genie. And that's kind of the layer that it sits at. And of course, Genie is bringing new AI capabilities to all roles, right? Maybe you're just a
[31:41] business user or an analyst. You just want to ask questions. I personally don't want to create a new agent if one already exists and is built in. So that's where Genie comes in super handy. Um, of course, there's also Genie code.
[31:53] This is more for technical users. And so think about data scientists, data engineers, and others. You're spending time building models, verifying which model is most performant for your use case. And so saving a bunch of time on
[32:07] that. And of course, Genie code is governed by Unity catalog permissions. So it can only access and perform operations that you have permissions for. Right? So that's the control uh in
[32:19] addition to context. Uh and so this really prevents any abuse or um prevents users who don't have access from using it.
[32:31] these videos are a little bit sped up, so you got to you got to pay attention in case you clocked out for a second there. Okay, so this one is I'm just asking Genie, hey, tell me what the revenue data is. It creates this chart
[32:44] for me, but what I really like is it tells me that, hey, in June there was a spike. It's automatically knows that I would care about that because of the context of my business. And so then when I ask, hey, like why was there a spike?
[32:56] It tells me it was because of this customer Apex Intelligence. But what's cool is it's telling me what it did in order to get that. And that to me is the true value ad, right? You're not going to trust an agent or AI without
[33:09] understanding it. If I have to go give this information to my manager or to my CDP, I have to understand how this data came to be. And the thinking of those steps is is super critical in that process. And of course, it can also call
[33:22] your existing MCP agents to uh complete the investigation even more. So that was Genie. There's also Genie code and it operates exactly the same way. So let's say you're a data scientist. You're
[33:34] looking to forecast the next six months of region-based sales for about like based on five years of history. Now for a data scientist, this is going to take time and energy and effort to do. But now they can just ask Genie code. Genie
[33:47] code is building the model here. It's telling you which model is is the most performant. And then it's also has drafting up the charts for you as a whole. you as a data scientist you can review and accept. So I feel like
[34:00] there's um this common talk in the industry that's saying that hey AI is going to replace roles and I don't think that that's true. What is going to do is replace the time that you are spending doing all of this work and making that
[34:13] easy for you. So you have more time as a data scientist and we are going to still need them to understand what's happening. At the end of the day, no company's g and say, "Hey, hold the AI agent liable, right? They're going to
[34:26] come to you and they're going to want to know exactly what happened." And that's where Genie and kind of the whole plethora of tool sets really comes into the picture because it's telling you exactly what is happening. Okay, now
[34:38] that I've given you the context of Genie, I wanted to talk about something that we launched at build just two weeks ago. So, Geniewide MCP aka the model context protocol for copilot studio is
[34:51] now in public preview. So, key capabilities of this is natural language access to the full workspace. Previously, you could only have it connected to a single endpoint, but now this is across all of your Genie
[35:04] connected spaces and across Unity uh catalog. So that per instance configuration is something you no longer have to do. And of course that reduces overhead significantly. So business users get that governed workspacewide
[35:18] data directly inside Copilot. Um and maybe I'm biased but I use Copilot a bunch and so this is super helpful because when my data is in Azure data bricks I can still easily uh connect to it. And of course like I'm not going to
[35:31] keep repeating this but goes without saying all of this is governed with Unity catalog right. So you're always only making sure that it's data that you have access to. So this was super cool and we announced uh like I said at build
[35:43] uh two weeks ago which was also an SF. Another thing we launched is um Ali I think did a good job of explaining the branching or how we no longer need uh production databases or copies of these
[35:56] databases for different roles and instead there's this branching uh that takes place to make it better for agents as a whole. And so that's effectively what lakebase is. We also announced uh lakebased branching with GitHub co-pilot
[36:09] agent mode. And so now when you create this copy on write branches of production um data, developers and AI agents can safely use them for building
[36:21] and validate them based on this data as well, right? But without impacting anything that's happening in production. So you can iteratively test fixes. you can more quickly detect anything that's going wrong or any data dependent issues
[36:34] and make your solutions really reliable and you can continue to iterate on them. So effectively you're creating a more efficient collaborative flow for building and validating data powered applications. So that's also in public
[36:47] preview and has been for the last two weeks. Okay, you've stayed with me all through September, right until right until uh two weeks ago and just today I'm super
[36:59] excited to announce this is my favorite announcement. Uh Genie in Teams and Microsoft 365 co-pilot. So the Genie app or the databicks Genie app is now available in Teams, bringing all of the
[37:12] wonderful things that I was telling you about Azure Data Bricks Genie directly into Microsoft Teams. So maybe your business users are not in Azure Data Bricks. Maybe it's still the data scientists. Maybe it's still the data engineers, but you're using Teams. At
[37:26] least I'm using Teams to talk to my co-workers. And when I was sitting in the keynote yesterday, I saw a lot of Teams conversations happening on Teams around me. Goes to show a lot of organizations are using Teams. So now we're bringing that context and bringing
[37:39] that insight and analytics to you versus across the board. Okay, apparently I need to talk faster. We're also launching Genie and Copilot's co-work. So that literally just went g yesterday.
[37:52] So co-work is enabling you to out uh kind of outsource your tasks and plan your tasks for you when you're not working. Um and now you're able to also leverage Genie for that. So basically in
[38:04] copilot co-work you can say hey Genie please do XYZ what are the NYC sales or any other question and it'll automatically connect you into Genie.
[38:18] Also all of the data and AI announcements that you heard available Azure data bricks day one that's how it fits into the entire ecosystem right so the new updated genie um ecosystem from
[38:30] day one within azure data bricks across the AI and and data workspace so speaking of fabric let's talk about the announcements on integration with Microsoft fabric and specifically one
[38:42] lake so Microsoft one lake is this unified organizationwide data built data lake built into the Microsoft fabric tenant. You're probably wondering, hey, that sounds a lot similar to Azure data bricks than what you just described. And
[38:54] what can I tell you? Humans, humans love choice, right? Maybe you've acquired a company that uses fabric or maybe your specific business unit needs to use fabric. You're not going to sit and move the data across the board. So, what do
[39:07] you do? There is this terminology called Unity catalog um of course in Azure data bricks. But how do you then get cataloges from external sources? That's known as catalog federation where you're
[39:20] taking catalog from something that's outside of Azure data bricks and bringing it into Azure data bricks to run the queries etc. All of the compute is in Azure data bricks and is not running on this external engine.
[39:33] And I'm super excited to announce today the general availability of Azure datab bricks one lake catalog federation. So just like the mirroring concept, it's just that in data bricks land, we call
[39:45] that catalog federation. Exactly like that, you have zero copy access to data that is within one lake directly from Azure data bricks. So no pipelines, duplication, data movement, all powered
[39:58] with that unified governance um as a whole. So no matter where your teams are, maybe they're in Azure data bricks, maybe they're in one lake, it does not matter. You can just effectively get the insights wherever it is.
[40:11] In addition, starting today in public beta as we call it, you can also store data just like you can in Azure data lake storage within one lake. Um, starting today for Azure data bricks,
[40:23] right? So again, zero copy persistence. Um, so any of your managed Unity catalog tables are directly in one lake without any additional pipelines or or duplication backed with that governance
[40:35] as well. And of course all of this in that open table format. You can write it in delta iceberg. It no longer matters anymore. But it's all available across the fabric engines without any additional work.
[40:48] If I had to summarize the fabric uh and and one lake integrations with Azure data bricks, I would just say it just works, right? I love this. I stole this from David actually where doesn't matter
[41:00] what your engine is whether you're using fabric whether you're using Azure data bricks wherever you're storing it whichever catalog you're using it just works right so you can be in one access
[41:13] your data without any movement you can be in the other access your data build your analytics without the other okay I think I've gotten all through all of that on time but I also just wanted to share um there's a bunch of
[41:27] announcements we've made in the last year. There's no way I could have fit all of those in 40 minutes. Uh we have a new blog space dedicated to Azure databicks on our tech community space. Check it out. It's just AKMS/ADB blog. Um and if you're more interested in the
[41:40] detailed features that we are launching and it's a long list, so be prepared to scroll. You can also find them on AKMSAB releases. Thank you so much for your time. Very much appreciated. And that's
[41:52] all I got for you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.