Skip to main content

Supercharge Your Enterprise BI: A Practitioner’s Guide for Migrating to AI/BI

Summary

  • Large-Scale BI Migration: This video showcases how Databricks migrated 2,000 dashboards in just two quarters from traditional BI tools to AI/BI without disrupting business operations.
  • Cost and Efficiency Gains: Learn how Databricks achieved cost savings and reduced reporting latency by moving to AI/BI on the Databricks Data and AI platform for democratized analytics.
  • User Adoption Strategy: Discover best practices for change management, user training, and adoption metrics to empower non-technical users with self-service analytics capabilities.

Supercharge Your Enterprise BI: A Practitioner’s Guide for Migrating to AI/BI

Watch: Supercharge Your Enterprise BI: A Practitioner’s Guide for Migrating to AI/BI
Talk by Aman Gupta, Senior Engineering Manager, Databricks ; RK Aduri, Staff Data Engineer, Databricks
Are you striving to build a data-driven culture while managing costs and reducing reporting latency? Are your BI operations bogged down by complex data movements rather than delivering insights? Databricks IT faced these challenges in 2024 and embarked on an ambitious journey to make Databricks AI/BI our enterprise-wide reporting platform. In just two quarters, we migrated 2,000 dashboards from a traditional BI tool — without disrupting business operations. We’ll share how we executed this large-scale transition cost-effectively, ensuring seamless change management and empowering non-technical users to leverage AI/BI. You’ll gain insights into: Key migration strategies that minimized disruption and optimized efficiency Best practices for user adoption and training to drive self-service analytics Measuring success with clear adoption metrics and business impact Join us to learn how your organization can achieve the same transformation with AI-powered enterprise reporting.
Intelligent analytics for everyone: https://www.databricks.com/product/business-intelligence
BI meets AI: https://www.databricks.com/resources/ebook/business-intelligence-meets-ai
Explore the only unified platform for agent systems: https://www.databricks.com/product/artificial-intelligence
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

FAQs

How many dashboards did Databricks migrate to AI/BI?

According to this video, Databricks migrated 2,000 dashboards from a traditional BI tool to AI/BI in just two quarters without disrupting business operations.

What benefits did Databricks achieve through migration?

This video highlights cost savings, reduced reporting latency, and improved user experience as key benefits realized from moving to AI/BI on the Databricks Data and AI platform.

Will AI replace traditional BI dashboards?

Rather than replacing dashboards, AI enhances them. AI/BI adds conversational interfaces and natural language querying alongside traditional dashboards, giving users multiple ways to explore data based on their needs.

What are the best alternatives to traditional BI platforms?

AI-powered business intelligence platforms like Databricks AI/BI combine semantic understanding with conversational interfaces. These platforms democratize data access while maintaining governance, moving beyond static dashboard paradigms.

Which BI tools offer AI-powered dashboards?

Modern BI platforms increasingly incorporate AI for natural language querying and auto-generated insights. Databricks AI/BI exemplifies this trend, combining traditional BI capabilities with conversational AI for a more intuitive analytics experience.

Full transcript

Good afternoon everyone. I hope all of you have had a chance to have your cup of coffee during the break because some of your dashboards might still need that caffeine kick. A quick show of hands. How many of you have actually thought that you could have brewed that second cup of coffee while waiting for your dashboard to load? or you just wanted a quick follow-up of why your sales spiked last quarter only to be met with a polite no from your analyst that yeah, we'll surely worked on it but probably next sprint.
If that sounds familiar, you are not alone. Traditional BI tools with all their hefty licensing fee falls short in the face of data needs of the modern digital organizations. So today we are here uh to talk about how we when faced with the similar challenges in data bricks decided to supercharge our BI experience through the organization by migrating to AIBI. I am senior engineering manager in data bricks. I have over 15 years of experience in data engineering and analytics and I'm joined with me.
Hi everyone, welcome to the session. I am RK Aduri. I have more than 15 years of work experience. I'm work currently as working as a staff data engineer as part of data engineering. Yep. So in today's session, we are going to talk about when faced with similar challenges, how data bricks decided to migrate to AIBI to democratize and supercharge our BI experience for all our enterprise users. And by the end of this session, you will have a ready to go migration playbook about migrating to AIBI and probably a nudge to start your pilot this quarter itself.
But before diving deep, let's talk about why modern organizations are looking to democratize data and AI. Today's organization realizes that it is not just enough to have selected few technical users to have access to all your data. They want everybody from field sales to finance ops to have the similar level of access as the analysts of the yester years. However, these traditional BI tools with their licensing fee which is prohibitive to grant licenses to every single users in your organization and then the need of the selected few uh analysts to help you make sense out of that data makes this a uphill task.
It is a stalling effort. But let's imagine it with a scenario. Uh imagine from marketing. She just has a simple ask. Uh why the customer churned spiked last week to get sense out of this. She turns to her trusted BI source to make a new report. Just to be turned off by that her license is not good enough to create a new report. She needs a new power user license which will cost her a arm and a leg. She went up her chain, got that finance approval.
She created that new report only to realize that her dashboard is taking forever to load. She needs to create a mini or extract to make it any performant. She went about that. She created the mini ETL uh dashboard started loading little faster only to be left with that there are multiple fractured metrics and a data silo which does not tally with her data lake house. Okay, it still works. Now she turned to her uh inbuilt AI assistant which came along with the traditional BI tool.
She asked the same question why my customer churned spiked last week? uh AI assistant replied, "Sorry, I can't answer that because you have not yet modeled your data perfectly." So in a summary, traditional BI tools with their prohibitive licensing fee does not let you grant license to everybody to begin with. Then the need of many ETL leaves you with a fractured data and another data silo and then the AI assistance being a secondass citizen does not let you have the same data democratization experience as you would wish for. to help answer that how AIBI challenges these status quo.
I have RK with me. Yeah, thank you. Okay, so let's find out if AIBI dashboards is the right strategic tool for your enterprise analytics. So we have like four strong points here to look at and evaluate whether the AIB dashboards are uh the right strategic tool. So the first one is their highly performant dashboards and you unlike your Tableau sorry uh unlike your traditional BI dashboards data and then maybe for faster refreshes and then you just wait for the data to load and so you you basically you end up uh consuming a lot of cost and so essentially uh you basic you sorry about that.
So as uh so you have the dashboards uh you need ABI's dashboards don't require any extraction of the data. So they're already available for you comes with the data governance and you have all the data that is needed with comes with the unity catalog and where this becomes the strong point is a they're cost effective that means you don't need to purchase any licenses a dashboard ABA dashboards doesn't have any licenses and you have an effective AI assistant called AIB Genie that I'm going to talk about in the next slide that is going to help you answer the questions without writing any SQL queries and all of this comes with unified data governance through Unity catalog.
So the Unity catalog is your go-to data governance model for all your data access needs and you don't need to come up with any other security models or access models to uh for AIB dashboards separately. So it all comes it's tightly coupled with data inence platform. So okay what is AIB genie? So let's say you want to have a conversation with the data. So this is essentially a chat interface that is right available right there in your dashboard.
So as you fire up your dashboard so this chat uh I'm going to call it as DA dashboard chat GBT uh we call it as like genie and this really works well with the human feedback that means you you can interact with it immediately and it it start giving you answers and also it understands your semantic model of your data. So the way it does is like the remember the unit catalog that I talked about. So all of the well annotated annotated data sets that comes with the unity catalog helps the genie to learn the semantic model of your data and then presents it in a chat format so that you don't have to write any dashboard queries.
You can just simply ask some questions to it and then it's going to start answering all that. So well uh the traditional BI dashboards don't um have very limited uh capability and then this is where this one of the strategic points where we decided to go with AIB dashboards the I'm going to talk about the performance as well like you can now think like okay so I have these dashboards and is it performant is it uh am I going to still wait for that coffee to brew and while this dashboard is loading.
So um essentially everything is serverless here. So you don't need to wait for the data to load and it optimizes very well with uh something called uh query caching. So the query caching is like it loads to your browser and then you can immediately have access to this data right there in your browser if it is small amp to fit in your uh browser and you can you feel like you're interacting with uh uh uh with essentially a real time interaction with your data.
So and what about the scale? Does it scale? Yeah, it definitely does. you have data into this platform that scales very nicely unlike traditional BA dashboards essentially you have uh you might hit a performance issue over there like it doesn't your extracts may be running very very slow and uh you might uh end up uh paying for the compute task and uh so this data intelligence platform scales nicely with unity catalog and everything and so that your AIB dashboards are primed for all the data analytics all your enterprise needs and then this is why it becomes your strategic partner for your dashboards.
So to talk about like how we all did that I I'm going to hand it over to thanks AR. So now that RK has shown how AIBI effectively democratizes the data and was the strategic choice for data bricks, now it was time to tackle our next big hurdle, which was how we can migrate 1,300 of our dashboards while keeping the business running. So repairing the plane while it is flying. So to tackle this, let me dive in deeper. So projects of this scales often stalls under their own weight.
To do this much more effectively, we decided that we'll uh start to think really slow during the planning phase so that we can execute fast without any roll back during the implementation phase. To do that we divided our whole project journey into five major pillars. We started off with inventory analysis so that we are not working against a moving target constantly. We started off by inventoring every single dashboard, its user uses and complexity and went about creating a very clean prioritized list.
In the second stage, we talked about creating a feasibility study where we documented all the equivalent features between the two tools and also any quirks so that every dashboard that we are migrating to does not feels like reinventing the wheel. You will have a ready templatized SOP to move from traditional tool to AIBI uh in a quick jify. In the third step, we uh invested heavily on our accelerator. There were three major accelerator we invested in.
One was reverse uh engineering where we were passing through the traditional BI tools BI logic and getting it into a clean spreadsheet so that you are not clicking multiple times to get through that logic. In the second stage, we talked about forward engineering where this logic was forwarded into AIBI and create uh help us create a complete UI. And the third accelerator was about creating a UI for the spreadsheet uploading.
You might realize that in most of the BI tools doing a ad hoc analysis on a spreadsheet is the most common COJ. We did not just want to replicate that CUJ but we actually wanted to enhance that experience so that uploading utility was a much better experience with a UI and now you have a genie on top of it. So instead of creating a new dashboard you can use LLM to chat with your spreadsheets. The fourth one actually was a uh regression uh uh utility where we were doing a data difference between the two tools so that the UAT users are not burdened with that heavy lifting of matching the data between the two tools.
Next stage it was about change management. Every new platform lives or dies by its adoption. We were very clear since the beginning that we wanted to engage with our stakeholders in the straightforward manner. We were talking about every bi-weekly steer co updates, targeted emails, demo invites and uh handholding sessions where we were enabling the users to make the transition much more easier and userfriendly. After doing going through this four pillars of thinking and planning, the fifth pillar was about execution.
The actual execution where we were migrating the dashboards from previous tool legacy tool to AIBI. Let's zoom in on each of these pillars further. The first step of our five steps migration plan was to uh get our house in order. We started off by doing a inventory analysis where we were tracking our uses based on metadata. So uh to do that there was admin dashboards in the legacy tool. We went about uh not only creating all the inventory list but also the users about uh from last months.
We were surprised to know that there were 84% of the dashboard which were not even touched since last several months. Uh we decided to ruthlessly retire them. This and this came with its own thorough change management where we were engaging with our stakeholders taking our qualitative feedback etc. But this gave us a quick early win where the pressure on us to get more additional licenses while we were migrating reduced much uh further and second we were left with a much more manageable list of dashboards which we could have migrated.
So if you are also thinking about migrating to AIBI but are overwhelmed by the number of dashboards that your organization has, a inventory analysis, a users analysis can be a very quick loweffort mechanism where you can come back with a very realistic timeline and a budget for this migration. Post retiring 84% of the dashboard, we were left with a linear set of 204 dashboards. to uh divide it further, we uh targeted the first 3% of them which was the more most complex and also the most used one.
The reason being if we could do the most complex one early on we would have all our learnings that would make the next 9% 117 of them like a cakew walk. Uh so this was our prioritization list from based on our quantitative analysis of users. But we did not rely just solely on this. This was also the time that we wanted to reach to our stakeholders and there were any number of that. We could not have possibly engaged with each one of them personally.
So we decided to identify champions among them. So these champions became a spokesperson for throughout our migration journey. They did not just uh bubble up any challenges from their BUS but also became the spokesperson of the centralized migration team that this is how the enablement would look like. We took their qualitative feedback on our inventory analysis and uh iterated over that list about how we are going about that.
So with uh our quantitative analysis and the qualitative feedback from our stakeholder, it was time for us to move to the next stage which was the building blocks. We did not kick off our sprint by migrating any single dashboard. We went about creating a infrastructure of building blocks which becomes a foundation layer. So it doesn't matter if you are going to migrate one dashboard or thousand it will stay for you. We started off this by uh creating a feature mapping and feasibility study where we documented equivalent feature between the legacy tool and AIBI along with it its quirks.
So a feature a from legacy tool always maps to a feature a sharp in AIBI. So thus creating us SOP or a template for us for the migration. In the stage two we moved on from theory to hands-on. This is the stage where we picked up the two most complex dashboard for each of the business units say GTM or marketing and created deployed this SOP to actually do the migration. Then we engaged our champions to iterate over the uh mig uh these new dashboards and uh created a new SOP as the feedback come came in.
By doing this we finalized the this is the SOP this is the sign off this is how we are going to do for all 122 of the dashboard that we're left with in the stage three I talked about the migration accelerator and tooling so by investing in these migration accelerator early on we were able to shave off almost 40% of the manual time in the subsequent sprints and we were doing like five times more dashboard compared to the first sprint the stage four this was a stage where we were talking about a centralized infrastructure.
I'll give you examples of say git checkins, schema uh creation, naming convention, alerting and monitoring. These are the things you cannot left your uh you can not left it to your views to do it uh again and again. These are the things that you have to do at a centralized organization place and then move on to translate it to the rest of the business units. This was also the time that we created a governance framework about the processes like raci framework between the analyst teams and the centralized operations team by uh with the scaffolding in place.
This was the time for us to step on the gas for the actual migration. How did we achieve the 122- boards in just 9 weeks? So uh these migration can last forever and going by my five pillars theory and all it might look little complex but if you focus on the actual time the actual time to deployment was 9 weeks. To do that we deployed a whole we divided the whole project journey into micro projects of two weeks each where we were going through a sprint.
We picked up a bunch of reports from our prioritized list. We deployed our migration accelerator to do the heavy lifting which over uh which migrated a dashboard with 80% accuracy. Then we deployed our SOPs for the rest of the 20% where we were bridging the gap uh for the fine-tuning. Next stage we deployed our uh regression accelerator where we were matching the data between the legacy tool and AIBI and providing a very simple report to our UAT users that everything looks fine.
Can you please sign off? Next stage we got the sign off from these UAT users and then most important deprecate the legacy tool legacy dashboard. If you keep on uh let it live, you will have new enhancement going on and it will never end end from there. So yeah, rinse and repeat 122 times till we get over our inventory completely. The second part here was be very clinical with your project planning when you go about migration.
This is also the time when users will ask for a lot more features. We were very uh intentional in our project planning that we wanted to keep the scope as is migration and but we also uh wanted to have it a iterative process. So we divided this through the agile framework where we were learning through our sprints and our future sprints were much more faster and much lesser uh QA defects compared to the initial ones.
Similarly uh don't rely only on the manual effort. Uh invest heavily in automation. That is how we had decided that the we did not want to start any of our migration till we have all our accelerator and heavy lifting in place. So we automated all the repetitive task uh so that our developers were actually focusing on the edge case scenarios and the quality of the output was much much higher compared to if they would have done everything manually with a repeatable uh clicks.
Similarly the regression end user population would be the highest for you. You would have thousands of users who would be owner of these dashboard. If you keep on waiting for the sign off from each one of them, it would take forever. You create a heavy lift through the this regression tools where you have a ready reports it is giving a green or red depending upon the data is 100% matching or not and maybe there are some ambers where you deep dive and figure it out that what was which row is not matching but it doesn't need to be done for every single of the dashboard.
So with the migration uh in place uh and a dashboard migrated it was time to move off our users from the legacy tool. So it is scary for the end users the business users to uh ac accumatize in the new platform and that is where clear communication change management and maybe a bit of healthy pressure also comes from the top. So we uh as I talked about we started off our change management much earlier than when we even thought about going about this migration.
So we kicked off our change management plan with targeted emails uh invites to our demo sessions where we were showing how our SOP is uh getting matched between the two tools and also a go links for all the frequently asked questions. In the second stage uh I talked about that we created our inventory but we also took uh the sign off and qualitative feedback from our champions and deployed these champions as our spokesperson for whole of our migration.
In the third stage we enabled our BU champions. We did not want to be the only centrally governed team who would answer all these questions. So they were bubbling up any issues that our users were having back to us which we were taking as a feedback and improving on that and they were also our spokesperson in terms that maybe this uh issue that you are raising uh is already solved for you and this is the way to go about that.
So you can't do it all alone. You need your champions, you need your team to be there for you. Fourth, uh we did not left it alone to the our champions to field all the questions. We went about a weekly office hours of inform, listen and redo which kept the confidence high because yeah, everybody want them to be listened. They were being heard and then we were also deploying that feedback if needed uh into our uh agile plan.
After going through this exercise, it was finally the time to revoke the license. If if you have given the sign off data looks good you and dashboard is migrated you cannot have the license throughout. So we revoked the license uh for that users for whom dashboard was migrated and mandated a companywide policy that every new dashboard has to be created in AIBI only thus cementing AIBI as the default platform for whole organization.
And after going through this, we also kept on tracking. If you remember, there was a users analysis dashboard. We created a similar metadata user analysis dashboard in AIBI. And we were tracking throughout that has the user really migrated is the adoption really picking up. And this became the centralized governance body to go through the whole migration. Let's see how this body took reigns next. So we have done the migration but how do you make any uh change like this any migration a lasting change where the people actually adopt the platform and don't just feel that it has been shoved down their throat.
So to do that we went about creating four pillars starting with the hyperare persona based training strategic tooling and centralized governance. In the hyperare phase when the dashboard was migrated, we deployed our thememes from the center COE and also our champions to hold weekly office hours where they handheld each of these end users even with the minor things like how to write their DBSQL queries or how to debug uh their newly deployed dashboard during the month and closer.
But we could not have done it throughout. We wanted to do it on the scale. So we then recorded 31 of these trainings which we deployed on our employee academy where people can go and do it in their own on demand training but we also did a lot of live trainings for the change management where a whole BU was invited and showed the journey of how AIBI looks like how the navigation looks and maybe a lot more persona based training where we were going really deep on how to create a DBSQL how to create your perform how to create a performance SQL in DBSQL with this.
The third part was strategic tooling. We did not just want the new tool to be a replacement of a legacy tool. We also wanted to provide that cherries on the top which was uh for in our case it was a spreadsheet analytics. We realized that uh 30% of our users were using spreadsheet as that spreadsheet analytics as the major CUJ in the legacy tool and it was cumbersome for cumbersome for them to upload this uh file back to the old tool create a new dashboard for just one users purpose share it back to their uh business users.
It was not so conducive but it was the major use case. To make it easier, we created a UI based utility where people were allowed to create uh upload their spreadsheet. And now Genie was the LLM based tool which did not need you to create even a dashboard. You can just start asking questions to your spreadsheet without even creating a new dashboard. Our end users love this experience. And if you are in the field of BI, you would realize that spreadsheet analytics has been a pain.
AIB does this really really well. Coming to the governance, so uh the analytics is generally driven by business. Centralized governance is still needed from compliance and regulatory purposes. So this is also the time for us to go back to the drawing board and think about the centralized alerting and monitoring. All your dashboards are not equally important but the one which are critically important should not fail and if they're failing continuously or their performance are dropping continuously this was the time for the COE to intervene.
So that is where the users dashboard and alerting and monitoring comes to play and also for the role level security data has to be secured. You can have only hierarchy based access to the data. All of this was done through Unity catalog and uh applying the role level security throughout the AIBI as well as data bricks layer. With that let's talk about our less lessons throughout this journey. Every major project like this comes with its own share of challenges and learnings.
We can boil down ours into three major ones. The first one was the feature mapping. people were accustomed of using the older tool and uh they wanted the exact look and feel also the UI based logic. To tackle this challenge, we uh migrated all the business logic onto the DBSQL layer for them. It has a two-prong purposes. First of all, DBSQL is much more faster. That is where we were talking about the real time and not waiting for your dashboard to load.
And second, it lets you do a much better change management through git control. If the logic is only at the UI layer, you cannot have a consistent change control, change management. While DBSQL lets you have the logic in your git layer and have a consistent change management. The impact we were able to accelerate the whole migration through the SOP based model in no time. I we have already shown it took us just 9 weeks to migrate whole data bricks onto AIBI.
Second layer was the change management people are accustomed to the way they are doing and any new change comes with the same stuff resistance to the change uh steep learning curve to handle that we focus early on about enablement. We did not want this migration journey to be only about all these dashboards are now in AIBI. You make sense out of it. We held 31 recorded and live trainings. We had user enablement in the minds right from the beginning.
We had champions who were the spokesperson throughout the whole journey. The result the early skeptics became our advocates in no time. And the adoption graph you will see in the next slide talks a lot about that. Third the project management the migration projects are generally the one which have to be very tightly controlled in the beginning itself. So analytics tool generally have tons of users. So stakeholder management and the consistent definition of done people keep asking for a new enhancement while you are migrating.
We were very intentional since the beginning that champions were the ones who would do the stakeholder management for us and the only definition of done is when the legacy tool is deprecated. You cannot have both the dashboards running and say the migration is done. So impact we were able to migrate 100% of the dashboard in just 5 months including the planning stage. With that you might say all good in theory but did it actually work?
We had we have spent last 20 minutes or so talking about how great AIBI is and how well we did our migration but if I were you I would have this question but did it really work? So yes let me let the KPI slide do the talking. Uh first the performance are the same most complex most used dashboard the top five of them were five times better performant in AIBI the chart below shows the stark difference between the two tools and the loading time.
Second uh about the adoption itself any new tool people have a lot of qualms about adopting to the new ones but we had almost the same number of users 2,000 of them uh ma the charts talk about monthly active users with a 80% higher NPS score. So yeah, people were skeptical in the beginning but when they saw that their dashboards was so much more performant and they don't have to wait for their coffee, they were ready to adopt.
The two inflection point that you see were the two major phases that we did where we uh deprecated the old dashboard and let people might start using EIBI. Third, this probably will ring a lot of uh bells with you that yeah, $880,000 of saving annually over just licenses. I'm not even talking about people, infrastructure or anything, just licenses. And this is for a relatively smaller organization like data bricks and number of heads.
Just see what this number would mean for your own organizations with tens of thousands of users. And all of this was done in just 5 months. So yeah theory does hold whatever we started off with our mission with the numbers uh do talk about that we were successful about it. So now what about your organization first of all migration is cheap fast and pays for itself in almost no time it takes time in quarters and not years I'll yeah the session might look that there were five stages and multiple journeys but all of this the act overall time was 5 months and the actual migration time was just 2 months for data bricks if we do intentionally and we follow this playbook.
Second, everybody is talking about AI right now and there are steps uh to reach there. AIBI is your fast pass to that. Genie comes inbuilt. Uh you can talk to your data in natural language and you don't need analysts to make sense out of that data. Your business users can use natural language to talk to this data and democratize it for their own self. And third, the establishing datadriven culture is no longer optional.
It was good to have some years back, but after the genai wave, it is a mustave otherwise your competitor is going to take that leap. So don't think tw you won't have to think twice about a licensing cost. That itself is a big win. And then you have a governance structure with UC right in place without doing another layer. So uh if you are ready to take that leap, if you're ready to start small, do your pilots, this is the time AIB is your strategic tool and we are there to help you out with that.
Uh we also want to thank our partners Leitics who Mik is here from that uh from the team to talk about that after this uh talk. They helped a lot with uh this whole implementation and you can talk to them. They have a booth in expo. Uh this is all uh forward-looking statement. We want this is forformational purposes. Uh next, please complete your surveys. It helps us a lot with curating better content for the next year.
If this session sound interesting to you, there are rest of these AIBI session for you uh which are going to happen today and tomorrow. And after this this is the time for any questions which we can take uh after this uh presentation uh down here. Thank you so much for your time and energy. Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.