Destination Databricks: Hertz’s Data Platform Modernization Journey

Summary

Hertz consolidated three separate data warehouses—Oracle, Redshift, and Teradata—plus two data lakes into a single Databricks Data and AI platform, eliminating conflicting data sources and centralizing governance through Unity Catalog.
The migration covered 527,000 vehicles across 11,000 locations in 160 countries and 26,000 employees, executing a phased approach with a 10-pillar de-risking framework spanning people, process, and technology.
Outcomes include real-time fleet operations visibility, support for advanced AI and ML forecasting use cases, and near-zero disruption during the final Teradata cutover using a lift-and-shift-first strategy.

Destination Databricks: Hertz’s Data Platform Modernization Journey

Hertz is executing a multi-phase transformation to decommission its 15-year-old legacy ecosystem and establish a modern data platform that powers BI and analytics across global reservations, rentals, billing, fleet management and enterprise operations. Hertz’s Databricks-powered data platform enables real-time analytics and enterprise-scale AI while improving data governance and significantly lowering operational expenses. In this talk, we walk through Hertz’s data landscape and our approach migrating from legacy systems.
Talk By: Lakshmi Nagesh Chinta, Solution Architect & Delivery Manager, Tata Consultancy ; RAM KRISHNAN, Principal Data Architect, Hertz ; Yared Gudeta, Lead Solutions Architect , Databricks ;

Chapters

FAQs

Why did Hertz consolidate multiple data warehouses onto a single platform?

Hertz operated three separate data warehouses—Teradata, Oracle, and Redshift—and two data lakes simultaneously, causing users to question which system held current data and which to trust. Consolidating into the Databricks Data and AI platform eliminated that confusion, lowered operational costs, and enabled centralized governance.

What was the scale and complexity of Hertz's data platform migration?

The migration encompassed data from 527,000 vehicles, 11,000 locations across 160 countries, 26,000 employees, and approximately 24 million rental contracts per year. Thousands of embedded applications depended on the legacy systems, requiring careful dependency mapping as part of the 10-pillar de-risking framework.

How did Hertz achieve near-zero disruption during the final Teradata cutover?

Hertz followed a lift-and-shift-first approach that prioritized moving data and workloads before optimizing for Databricks-native capabilities. Partners TCS and Data Switch, alongside the Databricks team, implemented a structured cutover plan designed to keep the business running throughout the transition.

What AI and analytics use cases did Hertz unlock after the migration?

Real-time reservation data now flows directly to fleet operations so vehicles can be repositioned as soon as a rental is made. The platform also powers advanced AI and ML use cases including demand forecasting, customer 360 views, and an operations metrics application for the executive office built with Databricks Apps and Genie.

Full transcript

[00:08] Okay, so to give you a context of the business of Hertz, um we operate 527,000 vehicles in about 11,000 locations across the globe in 160 countries powered by 26,000 employees
[00:21] writing about 24 million contracts rental contracts every year. So, in this session we are going to quickly talk about why we moved from our
[00:35] traditional warehouses onto the lakehouse platform. And we are going to talk about how we achieved it. What is the impact that we created?
[00:50] Okay, now the difficulty starts. Okay, the key drivers for transformation. Okay, the business had two big requests. One is they want to have real-time data for generating insights. For example, you know, as soon as rental reservation
[01:03] is made, okay, the fleet has to know so that they can do the fleet operation uh estimates and move the vehicles according to the rental request. Second is be able to
[01:15] power the same request for advanced AI and ML use cases so that we can do forecasting, customer you know customer 360, and applications similar to that. We had also developed an
[01:27] application for ops metrics for the CXO office using Databricks apps with the help of Genie. We'll talk about that. On the IT side, we had multiple
[01:39] databases. Uh three data warehouses at one point of time and two data lakes. Uh so, we wanted to consolidate everything together into one single data platform so that the trust in the data is higher. You know, and when we had Teradata,
[01:52] Oracle, and Redshift, users had to go to from one to other and they didn't know whether the data was current, and you know, which one to trust. So, by bringing everything into one single platform, we were able to kill that issue and operate at a lower cost,
[02:07] and also centralize our governance on a single platform using Unity Catalog. Okay, this is our uh how our data platform looks today.
[02:20] On the left side, we have all our data sources, both streaming and batch, coming from anything like IBM mainframes, Tandem uh for reservation, relational data stores, uh Kinesis streams, and MQ.
[02:34] Going through our uh medallion architecture, bronze ingested through SDP batch ingestion, uh SCD SCD2 on the bronze layer,
[02:46] and doing the data engineering uh in the middle layer, produce business insights for the gold layer. Uh data is accessible through both data warehouse and as well as for AI and ML
[02:59] use cases. On the right side, we have all the consumers. Um the modern applications uh created using Databricks apps, like on the one I was mentioning, the Ops Metrics for CXO Office, um
[03:13] building inquiries. There are several Databricks apps that we have created, and we also share data with our third-party partners through Delta Share as well as MFTs. We
[03:25] we are also embedded em AI and ML algorithms. For example, in customer-facing applications, like, you know, when you go to a reservation page, what vehicles appear based on your user preference, past history, other kind of,
[03:37] you know, clustering algorithms we use. So, when we wanted to migrate our the last leg, that's what we're going to focus today. You know, we're going to talk about
[03:49] although we We all our Redshift, Oracle, and two-data-lakes into the lakehouse platform. Today we're going to focus our last leg of migrating the Teradata, which is our largest data estate, and
[04:01] most complex, and most sensitive data, you know, into the into the Databricks platform. So, to start with, we had a very hard deadline of 7 months. You know, we kick-started the project in August, and it was due by
[04:14] February, end of February. That's when our contract ended. So, we wanted to migrate everything into the new platform. With near-zero downtime, because we have global user base, you know, so no user
[04:26] should know that we have moved from Teradata onto Databricks was the goal that we set. We had to comply all our GDPR and other
[04:40] regulatory compliances, because we handle customer data, sensitive data, you know, when you enter for a reservation, we do put your driver's license number, for example, it is sensitive, so it has to be secured. So, we had to make sure that our data is
[04:52] compliant with all the requirements from day one. So, it is baked into our migration process. Okay. Uh all these downstream users across the
[05:04] globe, you users that we came to know, you know, um were not there initially in our radar. We came to know while doing the migration, and we had to make sure that every user is
[05:17] covered, you know, their user requirements are met. So, they're embedded in about thousands of applications, I would say, you know, across the globe, applications that we didn't even know when we started.
[05:29] And, you know, we had to make sure the 15-year-old data that was in Teradata did not cause any issue for even like a one application somewhere around the globe.
[05:45] Okay. Yes, slide with just number five. So, what is What is number five? Any guesses? Why would number five become
[05:57] significant? We all have been through multiple migrations, right? We have application migration, database migration. We have been doing for like 30 years as when technology changes.
[06:10] But every migration, if you look at the war room, it will be chaotic. There'll be a lot of escalation, fixes, triage, right? Everybody will be on their toes.
[06:22] But not on this day, you know. Our war room was quiet. So, earlier in the day, when European users came online, they created about five low priority tickets. That's it. Rest of the day, you know,
[06:34] when you US users came online, the war room was quiet, totally quiet. At one point of time, I was wondering, you know, if our users were all on the system. So, I had called our director, you know, with a concern,
[06:48] "Check, you know, if users are on. There's something wrong, you know. It can't be like this." So, he said, "Okay, give me a few minutes. You know, your team check." And he called me back, "From everything, it looks all right.
[07:01] Business as usual." And that I thought is a significant moment because no other migration had such a zero, near zero,
[07:13] you know, issues in a complex data like migration. So, that's when I thought, "Okay, this story is worth telling." Okay. So, with that, I'm going to call Nagesh. He's going to walk you through
[07:27] how we did it, you know, right from day one, how it was planned for this success. You know. So, it is a team effort, obviously. Uh Team Hertz,
[07:40] TCS, and Databricks. It shows the how we were committed, the collaboration and competency of the team. You know, to pull through a feat. Come, Nagesh. No. So, I'll I'll hand it over to Nagesh to
[07:54] tell you the story of this migration which is filled with roadblocks, potholes,
[08:08] and unexpected turns. It a journey that is worth telling, quite memorable, to a destination Data bricks. With that, I hand it over to Nagesh. >> Thanks, Ram. Um
[08:21] I would like to take a moment to thank Hertz for giving this opportunity to work on a such a complex, uh, deeply embedded data warehousing system, which has been growing for over 15 years to,
[08:33] uh, move this into Data bricks, and we we've been able to successfully migrate it in 7 months. Also, with uh, the partnership with, uh, Data Switch team, Jared and Michael,
[08:46] it's been, uh, really wonderful working with them. Um, also, I would like to acknowledge the Data Switch partner, uh, we engaged for code conversion tooling, uh, so, that is another area where, uh, we
[09:00] worked with them and been able to move forward on this success journey. Okay. Uh, before I, uh, uh, get into the details, I would like
[09:13] to give a lay of land what we have been dealing with when we started. This system has been growing for over 15 years. It has deep integrations with rest of the enterprise.
[09:25] Users from across six 160 countries are using this for their daily reporting, decision making, partner exchanges for operational needs. On the ETL area,
[09:38] Informatica is having proprietary connections uh to sources and targets. Not an easy swap out. Schedulers are running from two platforms, Unix and
[09:52] Informatica. The data transformation logic is on three technologies, BTX, Unix, Informatica mappings. The data warehousing layer has the
[10:05] native integration implementations like uh temporal tables, set tables, multi-set tables, period data type, which needs the specific design to move to the new platform.
[10:18] Also, we have a set full set of consumers from BI, Cognos, Tableau, uh cubes, and internal and external data exchanges, and downstream applications.
[10:37] On this slide, I'd like to give the journey that we have taken. >>
[10:49] within the 7-month timeline. So, our approach is phase one, discover, inventory everything,
[11:06] classify all the uh complexity, and uh map the dependencies. On the phase two, evaluation, POC every unique pattern. Um bring users onto the platform.
[11:20] On the phase three, building and migration, wave wasted in delivery using multiple pod beams. And on the cut over
[11:33] uh prove the parity and then cut over to the new platform. These are the 10 pillars that we came up with across people, process, and technology.
[11:54] And these 10 pillars de-risk our entire uh entire program and gave the ability for us to uh deliver this within the timeline. So, with these 10 pillars, we created an
[12:08] environment where uh the migration will be successful. And I'll be going in deep of each of these in the next couple of slides.
[12:25] We looked at multiple approaches. Re-architecting from the scratch, incremental, domain by domain migration, hybrid, and lift and optimize. We debated all the options with the team and with all the stakeholders.
[12:39] And we locked the final option, which is lift and shift, as step one and optimize as step two. In step one, we want to do minimal refactoring
[12:52] and uh move the users uh to the new platform. And in step two, once we land, we want to work on the optimization uh techniques.
[13:09] Okay. So, as we finalized our approach, we got into the discovery phase, where we analyzed uh the entire tech tech technology stack,
[13:23] Informatica, Teradata, um um Cognos Tableau are users.
[13:36] We have done the data type mapping between the Teradata and Databricks. We identified interdependencies at uh job level and the domain level.
[13:49] Also, we have analyzed the users on the Teradata, the type of users, application users, the individual users, what type of operations they are performing on the warehouse. And then we scored each component at uh
[14:05] the complexity based on technical complexity, business criticality, and operational stability.
[14:20] And as part of this analysis, one of the key thing we found out is there is 15% of the code that was never been used uh and which is can be a candidate for discoping from the overall migration.
[14:33] So, what we did, we worked with the users. We uh uh did the screen test of the existing platform and we did not even migrate it. The other interesting thing we found
[14:46] here is we have found all the peripheral systems using this uh user analysis. And we have worked with them, brought them on to the migration program, and
[14:59] started working working with them from the discovery phase. Okay. So, having finished the discovery, we have two threads to start with. One
[15:11] is bringing users onto the platform. The second one is on POC every unique pattern. To bring users onto the platform, we need users account set up.
[15:24] We need users to be migrated start working on it. So, for bringing the users, we brought the roles and permissions from Teradata onto Databricks. Uh while doing that,
[15:37] we do have nested roles, which we had to flatten while migrating to Databricks.
[15:52] With this uh set up of uh users on uh Databricks, we were able to start uh looking into the permission pro- provisioning for uh the users and start
[16:05] with the next steps of uh uh providing the user with the data. Yeah. So, just setting up user accounts is not
[16:18] enough. We need to bring the data uh for make users start using the platform. So, what we did, we used Teradata native object store feature, copied the data from Teradata to
[16:30] Databricks, and then we have schedule sync happening nightly and hourly so that users can start using it. So,
[16:42] this helped us rather than users being in any traditional migration, we bring users at the UAT and the cutover phase, but we shifted that left, and we brought users onto the platform
[16:55] much earlier, which allowed us to upskill the users, uh which allowed us to help them with any um migration gaps, and we created user guides. We have
[17:10] uh um Uh we have also set up the uh workshops with the users in enabling their migration journey.
[17:23] And users started seeing the performance improvements much earlier than in any traditional program where they see at the end. So, the user adoption friction was completely removed
[17:35] by the middle of the program itself and it was very big success for us. So, coming to the POC, so, what we did is we have considered
[17:48] uh, every unique load pattern. For example, there are truncate loads happening, there are merges happening. Uh, there are append only, there are SCD type two kind of load. So, we took every
[18:00] load pattern, we took every uh, source technology, we took unique target technology. We have taken every unique workload and we have considered that for
[18:14] the POC. The end goal of the POC is we want to create a productionized code and a pattern that can be reused by our next phase, which is the development phase, build phase. So, we have B techs running from Cognos,
[18:27] we have B techs running from jobs. Uh, uh, uh, we have broadcast we got to do broadcast joins because there are smaller tables joining with the larger table. So, we got to do something different while migrating. So, during
[18:41] the POC phase, we have uh, done the identification of what is our target state solution should be and then we have used those for our
[18:54] migration. So, about 10% of the overall scope, which is remaining after de-scoping, was considered for POC. So, that helped us to to reduce the
[19:09] overall timeline by 1/3. If we would have not done the POC, this phase of the program, it would have taken three times more than what we have actually originally planned for.
[19:24] So, the new platform needs the new solutions. So, we have uh mainframe files uh uh that we need to be able to process into Databricks. So, we adopted Cobrix library.
[19:37] And there are fixed-width files, there are uh variable-width files. We got to process all of them. So, with Cobrix, we are able to process them at scale as well. Uh for mainframe DB2, we used the DB2
[19:50] Connect, and we are able to successfully set up a JDBC connectivity. For Cognos, we used dynamic query mode uh to get the desired performance.
[20:03] For say Salesforce, we used Salesforce bulk API to uh get the desired performance that we want.
[20:15] And for the IBM MQ, uh we uh created a bridge between the IBM MQ and our platform by creating a Python process which reads COBOL format files
[20:29] and drop a uh JSON message on the Kinesis stream. So, we got to come up with these uh unique solutions. Uh and these are few of the many other
[20:43] solutions that we have come up with for this transformation. Along with those, uh we have used uh internal accelerators for uh the data validation, test data setup,
[20:55] field mapping, and deployment validations. And one of the key thing I would like to highlight is the test strategy is one of the key thing. We have done the row level, cell level validation during the parallel run
[21:08] phase. And without this, it would have taken much longer than how we have achieved it.
[21:20] Validate at cutover. So, we have done the migration, but we want to be very sure that we are ready for the migration for the cutover. So, we have set up a lake federation lakehouse federation from Databricks to
[21:34] Teradata. With that setup, we have the access to do the live queries on Teradata Databricks to Teradata. So, we have the counts comparison happening, primary
[21:47] key comparison happening. We have created Databricks dashboards, and those dashboards we have uh been able to share with our stakeholders to get the sign-offs for uh moving forward with the cutover.
[22:01] And actual cutover is just a repointing exercise for us. We got to uh repoint from the NAS backed tables to the uh Delta Lake
[22:14] uh Delta tables. So, after landing, so we have started working on uh the uh
[22:26] op- optimization phase, where we started working on uh performance tuning uh by uh optimizing the workloads. Workload by workload, we have ensured that we are meeting the SLAs that were
[22:38] defined. And for Delta Lakehouse best practices, we implemented liquid clustering, optimize vacuum, and schema evolution disciplines. Quality metrics, we have implemented
[22:50] monitoring alerts, uh so that we have uh alerts coming out to our teams and the users. And also, we have brought in the FinOps discipline. Uh so, we have ensured the right size
[23:04] compute is set up and we have budget guardrails and resource tagging is happening at all the domain.
[23:28] Having completed the cutover These are the outcomes we have achieved out of this program. So, 1,500 new users have been unboarded
[23:41] from our legacy platform to data bricks. 15 AML use cases were unlocked. A couple of them include product order optimization and fleet optimization. We have started working on the near
[23:54] real-time data processing for one of the key domain, which is reservations. And for IT, 100% of workloads were migrated from data bricks. We have
[24:07] not renewed our legacy contract and there is a cost saving from the license and operational costs. Queries were executing faster
[24:20] on the new platform and the incident volume rate, which was reduced by 40%. We have a unified governance setup and
[24:32] single source of truth. These were the challenges of the CTO. So, those were enabled. And we have now the platform that we can use
[24:44] for experimentation at scale, which was not possible on the legacy platform. So, the last section, if you are in the
[24:57] journey of migration or if you have already uh uh or planning for it. So, these are the six takeaways we want to give you. Uh lift first, optimize second. Do not
[25:09] mix both. Two different problems, migrating and optimizing. So, that it uh which increases the risk of migration. The second one, early user adoption. Uh
[25:22] so, consider users as one of the key assets to move along with you. Uh rather than do not consider any migration projects as an IT uh driven solutions. Instead, bring your
[25:35] users along. POC every unique pattern. The learnings from the every unique design well in advance uh will uh help a lot, which uncovers a lot of
[25:47] uncertainties in advance rather than wait until the end. Maybe we may see early success if POCs were not done, but uh the tougher issues will come out if every unique pattern is
[25:59] uh POC'd in advance. Having uh governance on day one uh will really help rather than considering the governance as a post-migration cleanup
[26:11] activities, which we traditionally see in many other migration uh projects. So, this enabled us to have a production-ready system when we cut over. Leverage accelerators. This is one of
[26:23] the key thing. Uh This has helped us accelerate. And also the front-loading the risk, uh ensuring the toughest uh workloads are done in advance and
[26:35] having a structured team uh to have the uh domain level uh definitions or domain level scoping
[26:47] defined for the team, delivery teams. Rather than having a big migration team, having a uh smaller teams will really help in the migration. Yeah.
[27:02] So, thank you. And with this, we'll take your questions. >>

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.