Enterprise Data Transformation: Databricks, Governance, and Adoption

Summary

GM Financial completed a three-year Databricks migration on Azure across 17,000 tables, 1,300 users, 1.5K pipelines, and 2 million monthly queries by establishing an enterprise data strategy board and a Databricks Center of Excellence.
The team drove adoption by gamifying training with community challenges and a leaderboard, and deployed Genie-powered self-service analytics environments alongside FinOps practices for cost visibility and optimization.
GM Financial's Data Evolution 360 playbook demonstrates that successful data transformation requires institutional commitment and culture-driven enablement alongside the technology migration to maintain SOX compliance while enabling innovation.

Enterprise Data Transformation: Databricks, Governance, and Adoption

Enterprise data transformations stall without strong governance and people strategy. GM Financial's three-year Databricks migration on Azure demonstrates how institutional commitment, beyond technology, drives success at scale. Managing 17,000 tables, 1,300 users, 1.5K pipelines, and 2 million monthly queries required rethinking the entire data operating model.
this video shares GM Financial's Data Evolution 360 playbook: establishing an enterprise data strategy board, building a Databricks Center of Excellence, gamifying adoption with community challenges, deploying self-service environments powered by Genie, and implementing FinOps for cost visibility. Learn how top-down governance, community-driven training, and organizational agility maintained SOX compliance while enabling innovation.
🤝

Chapters

FAQs

What is the Data Evolution 360 approach GM Financial used for their Databricks migration?

Data Evolution 360 is GM Financial's multi-faceted transformation playbook combining technology migration with organizational and cultural change. It includes establishing an enterprise data strategy board, building a Databricks Center of Excellence, gamifying adoption through community challenges, deploying Genie for self-service analytics, and implementing FinOps for cost visibility.

How did GM Financial drive adoption across 1,300 users during the Databricks migration?

GM Financial gamified training by introducing community challenges and a leaderboard that recognized employees who deepened their Databricks skills. This community-driven approach complemented the Center of Excellence's formal training resources and created organic enthusiasm for the new platform alongside structured enablement.

How does GM Financial use Databricks Genie for self-service analytics?

GM Financial deployed Genie-powered self-service environments that allow users across the organization to ask natural-language questions about data without writing SQL or involving the data engineering team. This capability is part of their broader strategy to move from centralized data delivery to an empowered self-service culture.

How did GM Financial maintain SOX compliance while migrating to Databricks?

GM Financial embedded governance into the migration from the start, with Unity Catalog providing access control, lineage, and auditability across all 17,000 tables and 1.5K pipelines. The enterprise data strategy board provided top-down oversight, while approval gates in deployment processes ensured that changes to production systems met regulatory requirements throughout the three-year journey.

Full transcript

[00:08] Welcome everybody to today's session, which is the essential ingredients for a successful data modernization program. Uh if you are in here to hear more about how GM financial modernized their data, uh this is your session. Uh so by way of
[00:23] introductions, I'm Lata. I have 28 plus years uh leading data programs at scale. Uh and uh over several years, I have basically been a trusted CXO advisor. Uh I have done several data migrations
[00:39] even before cloud was a thing. I moved data from Oracle to SAP, SAP to um Microsoft Azure and different other platforms. And so, the journey continues and here I am 28 plus years later.
[00:54] I'm also the executive board member of CDO magazine, and I'm a thought leader on data, AI, and analytics. And I do a lot of speaking engagements around the country, and that's what keeps me hopping busy. Uh outside of work, uh for those of you
[01:11] who don't know, I also like to dance. I'm a trained classical dancer. I've done that for um over 30 years, and I'm only 18 years old. Uh and I also like to travel the world in my spare time and uh spending time
[01:27] with my family. And in there, you see some of um my awards and industry recognition. I was on the CDO top 100 two years in a row, CDO top 40 financial leaders uh in 2025,
[01:43] and CDO magazine top 100 global data women uh in 2024 and 2025. Um my favorite quote from Martin Luther King that defines me, and I spent a lot of time telling my leaders this is if
[01:58] you can't fly, then run. If you can't run, then walk. If you can't walk, then crawl. But whatever you do, keep moving forward and don't waste any time because time is of essence. And this stays with me and some of my
[02:15] leaders who are in the room today have probably heard this again and again. Um So, let's talk about transformation. What's the difference between change and transformation? And I picked up this quote by Nick Candito which says that companies that
[02:31] change may survive, but companies that transform thrive. Change brings incremental or small-scale adaptations while transformation brings great improvement that ripple throughout the future of an organization.
[02:46] And I think that's very apt for what I'm going to be talking about for the next um hour today. So, um that's going to be the focus of my presentation. So, by show of hands, how many of you here,
[03:03] when you were growing up and your parents called you when you were out on the playground playing and they said, "Come quick, we have a dessert waiting for you." They didn't say dinner, they said dessert. And you rushed up and you went upstairs to eat
[03:20] the dessert. Raise your hands. I'm going to raise my hand because I was one of them, right? Okay. So, now you go home and your parents show you the spinach, right? And you're
[03:35] like, "Oh man, I came here to eat the cupcake, but they gave me spinach." But this is the same analogy. The I equate the cupcake to AI, to innovation, to all the great things you do to keep your companies, to increase
[03:52] your operational efficiencies, increasing revenues. But the spinach, it is your data transformation journey. You have to transform your data. You have to improve data quality. You have to have a business glossary. And those
[04:08] are some of the essential building blocks and the foundation so that you can enjoy your cupcake, right? And so, I'm still suffering from the PTSD from my childhood where my parents would
[04:24] cheat me by asking me to come eat cupcake, but it was a bowl of spinach. So, um with that analogy, I would like to take you into my presentation today.
[04:39] So, when I started at GM Financial 3 years ago, uh on this journey, I put forward a vision that consisted of five pillars. My first pillar focused on data privacy and enablement. How do we keep our data secure and trusted and safe?
[04:55] Data modernization, which is how do we organize our data, build a business case, and transform our data so that we can innovate and and and do AI at scale. How do we set up a well-sustainable
[05:11] foundation with the enterprise data governance framework and an operating model? Strategic analytics and AI launchpad, which is how do we consolidating how do we consolidate the three or four reporting and analytics tools we have
[05:27] and come to one one tool stack, one tech stack, and also build the AI center of excellence, and then business insights enablement, which is how do we make sure that all data all data is at the heart
[05:43] and center of all of our application modernization initiatives, data is not an afterthought, that it is actually something you have to think about every time you modernize an application. So, that is where we started 3 years ago
[05:59] on our journey, and we basically built our business case in 2024, and we started our program in the end of 2024. Our program for data modernization is called Data Evolution 360,
[06:17] and the name came from basically evolution came from the word project evolve, which is GM's EV vehicle program, and 360 came from the 360° view of your business data.
[06:33] So, a little bit about GMF and what made our migration very complex, we actually started the migration in the end of 2024, and we are actually wrapping up our migration at the end of 2026.
[06:49] What made our migration very complex was that we had we went from six different disparate sources of data in Oracle to one lake house on the cloud. Secondly, we had several different
[07:04] versions of the truth because our customers had copied the data from applications and moved it into what they called as sandboxes, which was nothing more than just Oracle schemas that did departmental reporting.
[07:20] And thirdly, they had taken the metrics they provide they produce in their sandboxes and distributed and cross-shared across the company. And four, our data was in a data vault, and we had to unvault the data to bring
[07:37] it to the cloud so we can do it efficiently. So, we've been on this journey for the past 1 and 1/2 years, and I'm very excited for Dec- December of 2026, where our journey is going to be coming to an end. And we are really looking
[07:53] forward to innovation after that, increasing revenues, improving operational efficiencies, looking forward and what's next in terms of speed to market and innovation, and really helping our customers
[08:08] understand a 360° view of their business data. So, today's presentation that I have on how we modernized our data, I'm here to tell you technology is just the is not just the silver bullet. You
[08:25] have to think about the world from a people, process, and technology perspective. So, to that end, I wanted to introduce the concept of agility, proficiency, and communities on which today's presentation is based on, because I'm
[08:42] going to cover what we did in each of these facets to to continue our migration and successfully plan towards a targeted end date of December 2026.
[09:00] Before that, I wanted to get a little bit into our legacy technology stack. So, from a legacy perspective, we were an Informatica on-prem on-prem ETL tool on Oracle Exadata. We used SAS for everything, for ETL, reporting,
[09:16] analytics, data engineering, everything. Um Oracle Exadata, and then we had a plethora of reporting and analytics tools. We had Cognos, SAS, Excel spreadsheets, which I affectionately call as spreadmarts, and Power BI.
[09:35] And in the new world, we consolidated, streamlined, and came to Databricks PySpark, Databricks on Azure, and then uh we are actually working on deprecating our legacy platform, Cognos,
[09:53] in this year. And we are also looking to basically move all of our analytics and reporting users to Power BI and use SAS for its intended purposes, which is machine learning, predictive modeling,
[10:08] and all of that, and not for data engineering, for which we have PySpark SQL. We also have an enterprise data business glossary uh and data quality platforms and MDM with Informatica. Uh
[10:23] from a uh So, what does our lakehouse look like? So, our lakehouse has uh bronze, which has the raw data, and then we have the silver, which has the cleansed data, and then we have the gold, which is the uh enriched curated
[10:40] data. And we also have this concept of self-service environments because remember I told you we had sandboxes that our customers had built, which were nothing more than Oracle schemas, where they did departmental reporting. We
[10:55] transitioned all of those business partners into self-service environments. And we put together a lot of governance, which I'm going to cover later, so that we could lock down some of the governance, we could infuse some
[11:11] standards, and make sure that we have some best practices governing these self-service environments. So, data evolution 360 at a glance, uh total number of tables 17,000.
[11:27] We have about 1,300 users. We have about 2.3 petabytes of data per month. We have about 1.5 K pipelines, 2 million monthly queries, and we have about 500 AI interactions per month.
[11:46] From a Databricks features perspective, data engineering, governance, and security, SQL and analytics, machine learning, GenAI, and sharing and federation, these are all the features that we use. Um we have from a data engineering
[12:03] perspective, Lake Flow declarative pipelines. From a governance perspective, Unity Catalog has proved proven to be a great product for us where we can see the end-to-end technical metadata. Databricks SQL and serverless SQL, which
[12:19] has been very beneficial to us. Um and we've been able to put AI BI dashboards and Genie spaces to best uses. And we do a lot of work with machine learning with all of the features noted in the presentation too. And of course,
[12:35] from a sharing and federation perspective, Delta sharing is of immense value to GMF because we can now start to shake hands with GM sharing data, and basically providing data to GM uh so
[12:51] that we can have a complete holistic view of a GM customer who walks in the in the showroom, buys a car, and then comes to GM Financial to f- to finance their loan. So, that's one of the projects that we are working on with GM
[13:06] right now to facilitate Delta sharing. We also use foreign catalogs pretty heavily. So, what's on the horizon? Um we are looking at Lakebase. I'm very excited for the future with Lakebase.
[13:24] Uh we have a pilot that's running for LakeBase, and um I'm uh very excited for the future of LakeBase because it allows us to do near real-time uh data warehousing. And so, that is one thing that's very exciting
[13:40] on the future. Also, looking at AI agents and Databricks apps because we have a lot of Close your ears. Access databases at GMF. And so, we're looking to in uh some of
[13:57] our international operations use Databricks app so that we can sunset those Access databases, those Excel spreadsheets, and use Databricks apps to help uh improve operational efficiencies.
[14:21] So, speaking of communities, um as I said, the three concepts of community, agility, proficiency. I want to focus on um communities. And from a communities' perspective, we set up a very well-established
[14:38] enterprise data governance framework. If there's one thing you take away from this session today, if you're planning a cloud migration, please, as soon as you build your business case, establish an enterprise data strategy board. You need
[14:54] to have a scalable enterprise data governance framework that is actually helping you push decisions top-down in the organization. So, in our organization, we set up an enterprise data strategy board
[15:09] uh with our CEO and CIO as chair and co-chair. Several members of our executive team sit on the board, and we meet quarterly, and make and there are a lot of decisions made about data governance,
[15:26] data policies that impact GMF, uh new regulations for which we need data mapping, data literacy. So, there are a lot of important decisions. This board also governs the data evolution 360 product road map.
[15:48] One level down from the enterprise data strategy board, we have a data governance office which is a cross metrics representation from cyber, data, compliance, and security, which meets bi-monthly. And um the folks who are on this council are
[16:03] actually advisers to the enterprise data strategy board. And they prepare the agenda and that's what is input into the enterprise data strategy board. We have eight different business data steering committees that meet either bi-monthly or quarterly. We
[16:21] have a lot of MBRs and QBRs in uh Latin America, in Mexico, Brazil, uh Chile, Colombia. We also have several in North America, where we meet with our customers and give them program updates,
[16:37] portfolio updates. So, all of the data initiatives are front and center and they are making key decisions uh at the leaders level. We also have a data stewardship council, which consists of a cross representation of several
[16:54] business units. And the data stewardship council is very instrumental in leading and managing the business glossary, making sure that the data owners are actually approving data and making the right decisions related
[17:10] to their data. We also have a community of practice uh in uh we have set it up all over the globe and most recently we set up one in Brazil uh where we started it in the local language in Portuguese. We have
[17:27] one in Mexico uh which is in Spanish and the folks in Colombia attend that too. So, we want to make sure that we are staying close to our customers. We are educating them on this journey. We are helping them embrace the products and
[17:43] tools which you will see later in the slide. So, moving from communities uh I wanted to focus on proficiency. And in proficiency we uh I wanted to talk a little bit about how we built our capabilities
[18:01] through training structure and hands-on adoption. So, we built a Databricks Center of Excellence about a couple of years ago. Many thanks to Databricks for bringing all their resources to help us make it a success. And we also started
[18:16] an adopt-a-thon which I'm going to talk about. So, our metrics today so year to date uh we have 400 and so the four 4,386
[18:31] is cumulative. I think in this year we have about 100 and 1,200 participants who are trained across all uh sections. We've had 21 sessions, 69 training hours delivered this year year to date and
[18:48] 8,125 man hours dedicated to knowledge building. Our motto uh having data isn't enough. You succeed only when your business embraces your data.
[19:04] So, to that extent like I was talking about we built a Databricks Center of Excellence where we focus on onboarding and training. We provide support and assistance. We also uh have office hours, which is monthly uh twice a
[19:20] month. We have innovation and best practices. Um we share a lot of best practices. We have a wiki page where we propagate all of the latest update. Mission perspective, we work very closely with Databricks who does a lot
[19:36] of Databricks days at GMF. And we have sessions where we work with them to basically uh generate some excitement at GMF with our business partners. Uh performance monitoring, cost monitoring, we're going to get into that
[19:53] in a bit. And of course, knowledge sharing. So, we are a car company as you would have realized uh in the past 18 some minutes that I've been talking. So, we believe that we are going to use
[20:09] the uh analogy of Formula F1 series to show where we are in our adoption journey. So, to that extent uh our adoption race track uh we have the milestones of enablement, which is we
[20:24] train our business partners. We have several defined curriculum that we have built in-house. We have uh sessions on Friday where we where we have uh we we have study groups and some of our business partners come there so that
[20:41] they can study, attend the exams, and get certified. And then of course, the requirement scope, implementation, and go live. So, we've gamified the whole Databricks adoption because migration is not going to cut it. We need to complement that
[20:57] with adoption. So, our adoptathon, this is you have a marathon, this is an adoptathon. So, we help our customers learn and certify. We help them migrate and deliver their data
[21:13] to the cloud. We help them climb the leadership board. So, what we have done is we have gamified it by breaking down the adopt-a-thon by various leaders in the business units, and I have a sample
[21:29] dashboard here to share with you. But, each leader's team, as they finish certain milestones, they get certain celebrating you points, which they can then take to the GM store and cash those points for any rewards they want. Either
[21:46] it could be a sweatshirt, a hoodie, or whatever the case might be. It could be a iPod charger or something like that. So, we encourage them by giving them celebrating you points. Um and so, that's how people who get
[22:01] trained, who migrate their data, they climb the leadership board, and they get recognized uh with bonuses and uh the celebrating you points. So, this is an example that I I put together that was in one of our slides.
[22:17] If you can see the first one, it shows all the leaders, our EVPs, SVPs, who work at GMF. Uh and some of them are here in the audience today. Thank you for being here. But, this shows how we break uh our adoption journey and our
[22:34] adopt-a-thon by points. So, if you are trained, you get a certain level of points. Um if you certify, your your points go up. If you migrate your own data and you show a success story, you get some more points. So, it keeps
[22:50] adding up that way. And this has generated a lot of excitement uh with our businesses. They are very excited, and I think everybody has been very enthusiastic um you know, with the concept of gamification. So,
[23:06] I would highly encourage everybody, and we can talk more if you have questions. So, adoption journey, we do a lot of learning and enablement. So, it's very important to know that we don't keep our learning and enablement
[23:23] only focused on Databricks. We also help our customers understand Power BI, what is changing with Power BI, and Power BI fundamentals, Python transition,
[23:39] for SAS users. And we also, what we do is because it's hard to remember all the Databricks fundamentals, we also created bite-size videos. And then the second swim lane is we have requirements and scope.
[23:56] We also work with our customers first to understand all their requirements and scope of all of their sandboxes that need to move to the cloud. And then we do an environment readiness, where we make sure that they have all the AD groups set up, the network
[24:12] connectivity. And when they are ready to move their data, that they can actually move forward with a high degree of velocity. And then of course comes the implementation, and finally the go-live and sign-off.
[24:33] So, next up, I wanted to talk about agility, which is the last pillar, basically. So, here we are going to talk about how we democratize data assets. I can understand when you have a complex data migration running, you have to give your customers the
[24:50] ability to basically have some space where they can learn to use Databricks, where they can experiment, where they can innovate, where they can come up to speed so that
[25:06] they can help you as well with moving their sandboxes. So, we created this concept of self-service environments, and we also introduced Genie. And the second one I wanted to talk about was we also built a scalable FinOps practice, and so we're going to
[25:23] cover that in detail. So, we put together this concept of self-service environments. Remember I talked about the sandboxes? So, all our customers are now in the process of converting their sandboxes into self-service environments.
[25:40] And this allows users to join their own data with enterprise sources for reporting. It gives them access to tools like Genie where they can quickly answer routine questions. The metrics stay within the department,
[25:55] so we are locking down the self-service environments. No organization is allowed to take the metrics from their self-service environments and share it with the rest of the organization. It has to be in the enterprise lakehouse. And we've built certain standards and
[26:13] policies so that we can govern and implement best practices. So, we have a good success story to say. Reflecting back last year same time at the Databricks conference,
[26:30] we had several hiccups with Genie when we first introduced it within the organization. But Genie has really taken off, and it's really created a space for itself at GMF. A lot of our customers use
[26:46] Genie for AI-assisted coding, and we've had about 500 plus AI users across platform with an average of 1,000 prompts per person and we have surpassed 500k directions in March and April. So, Genie
[27:04] is actually working very well for us. Of course, I heard the bad news today that we're going to be charged for Genie going forward, so not happy about that, but um So, Genie's been a great success story
[27:20] for us. We really enjoy it and I think it really has made our lives easier with all of the prompts and queries. We also have FinOps. We set up a FinOps practice about two two years ago so that people can get measure cost versus
[27:37] performance by having visibility into the costs at a pipeline level. Ability to drill down at workflow, task, cluster, and at a use case level. We also started on a showback to
[27:52] business both Azure and Databricks. We don't do chargeback, but we do showback. And just in this year, we went from all of our traditional dashboards that were in Power BI for FinOps to
[28:08] Databricks. So, we use the Databricks native capabilities for dashboarding for FinOps. And then the benefits of FinOps, of course, is the pipelines and workflows. We can do it by use case, which is
[28:23] understanding at workspace level consumption, subscription consumption summary, task level cost metrics, top 10 jobs by workflow, and um we have some data police in our in our
[28:39] organization who are probably in the audience here who call people and say, "Hey, your query was $200 today." So, uh I know people are paying a lot of attention to it. SQL warehouse, so we pick and choose the use cases by tags to
[28:54] understand the cost. It's best programming practice that you have tags as you go to production so that you can you can basically follow suit and have be able to do FinOps effectively. Better understanding of cost cost to
[29:10] performance ratio. And actionable insights that we were able to gain was the SQL warehouse consumption and cost. Query history, execution time, number of queries by user and warehouse, number of
[29:26] queries per day, price per query. So, I wanted to spend a few minutes here. We talked about agility. We talked about proficiency. We talked about communities.
[29:41] And really at the heart of our complex data migration was the people, the process, and technologies. And while I am here to tell you technology is not the silver bullet, it
[29:58] is not the silver bullet. There were several things we did which if you are planning an extensive cloud migration, I encourage you to think about these things. First one is innovate for growth. You have to have hackathons, which is
[30:15] what we did. We encouraged internal hackathons so our team members worked on frameworks, accelerators. We had one person who worked on accelerator to un-walled the walled and bring it and unpack the code to straight SQL. So, we had some innovations from
[30:32] within our very talented team of data engineers. Upskill your teams to prepare them for for journey. So, one of the things we did was we bought a Databricks training plan and we trained some of our team members uh and brought them up to speed
[30:50] on Databricks. We also created curated training programs. And the third thing we did was we created this concept of Focus Fridays where we clear our team members' calendars from 12:00 p.m. to 4:00 p.m. so that they can focus on deep
[31:06] research, learning Power BI, learning Azure, getting certified, prepare for the exam. So, these are all small things that go a long way in helping your team members upskill and prepare them for this cloud journey.
[31:23] We also established data governance framework and built a very well-established data catalog. I would say data governance is the key to a business data transformation. You want to transform your data, you want to
[31:39] make sure that all your decisions are moving top-down in the organization. Otherwise, you're not going to be able to succeed. Um next up, customer engagement with new technologies. Uh
[31:54] we really worked with our uh business users. We brought data days to them. We partnered very heavily with Databricks to have Databricks days. Uh we had several sessions with with them. We have a Datathon.
[32:11] We have something we're moving to this concept of Adopt-a-Tell. It's like show and tell. So, most of our business partners who have assets in production in Databricks, they actually come to our office hours
[32:26] and showcase their accomplishments and talk about how they do it and give others an opportunity to ask questions. Um and next up, I would say um organization change management. So, we really handled it two ways.
[32:43] First one is, I believe that uh I heard somewhere where it where it was said, you have to say the same story six times or seven different times before people understand it. So, which is what we do. We use a lot of the company's internal
[33:00] communications department. Uh we've built our talent and skill sets and OCM. And we have continuous newsletters, SharePoint articles, Wikipedia, ex- etc. that take us uh
[33:16] that help us every step of the way in disseminating the information across the company. The other thing we did was before we started our migration, we decided to go from an organization
[33:33] which had about 220 some people that was completely waterfall to being completely agile. And that was a essential foundation for us to get to where we are because um it helped us get traceability
[33:52] and visibility into what our team members were doing. We were able to organize ourselves in pods. We were able to uh dedicate a lot of time to data evolution 360. And so, uh
[34:08] moving to an agile methodology has been the key to our success for our uh cloud data transformation journey. And um I wanted to say this. So, this brings me uh to the end of my presentation uh with
[34:25] a very profound statement which says, "Alone we can do so little. Together we can do so much. This journey has really It takes a village, and the village was uh basically so many of our team members
[34:41] internally, a very talented team, uh and our business communities, our subsidiaries, our GM partners, and everybody who helped us on this journey. And I wanted to take a few minutes today
[34:57] to thank my teams who are in the audience and the ones who are not here today because they have done a phenomenal job of bringing us to this area and to bring us to what success looks like. I'm really counting down the steps to December the days to December
[35:14] 31st so that we can be done with this migration and take a victory lap. I would also like to thank all of our business partners who are in the audience today, Alan, Santosh, uh Blake, and uh Flavio,
[35:31] and um Michael Besti, and everybody because they are the people I call every day. They are the people who are my willing partners. They've always been very enthusiastic champions of what we do. Uh and I really encourage and I really
[35:47] appreciate their support, their encouragement, and everything they do. And last but not the least, I wanted to take a few minutes to thank our Deloitte partners who have been joined with us at the hip uh over this past 3 years, been on the speed dial, and uh patiently
[36:05] answer answered all my questions. And um that's about it, and that's a wrap. Thank you so much. I hope you enjoyed uh my presentation today.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.