Three Big Unlocks to AI Interoperability with Databricks

Summary

• AI acceleration hub: Learn how Databricks acts as a central hub at Expedia for scaling model training and prediction generation.
• Unified ML platform: See how Expedia eliminated silos by building a unified ML platform anchored on the Databricks Data and AI platform.
• Cross-platform synergy: Discover how seamless integration with Expedia's ecosystem accelerates time-to-market for AI-powered travel experiences.

Three Big Unlocks to AI Interoperability with Databricks

Talk by Shiyi Pickrell, Senior Vice President, Expedia
The ability for different AI systems to collaborate is more critical than ever. From traditional ML development to fine-tuning GenAI models, Databricks delivers the stability, cost-optimization and productivity Expedia Group (EG) needs. Learn how to unlock the full potential of AI interoperability with Databricks. AI acceleration: Discover how Databricks acts as a central hub, helping to scale AI model training and prediction generation to deliver high-quality insights for customers. Cross-platform synergy: Learn how EG seamlessly integrated Databricks' powerful features into its ecosystem, streamlining workflows and accelerating time to market. Scalable deployment: Understand how Databricks stability and reliability increased efficiency in prototyping and running scalable production workloads. Join Shiyi Pickrell to understand the future of AI interoperability, how it’s generating business value and driving the next generation of travel AI-powered experiences.
Databricks Named a Leader in the 2025 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms: https://www.databricks.com/blog/databricks-named-leader-2025-gartner-magic-quadrant-data-science-and-machine-learning
Build and deploy quality AI agent systems: https://www.databricks.com/product/artificial-intelligence
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

FAQs

What ML challenges did Expedia face before adopting Databricks?

This video reveals that Expedia had 10 different ML systems operating in silos, with each team maintaining separate Databricks instances. This fragmentation caused teams to rebuild models, duplicate efforts, and lack standardized deployment practices, making scaling and collaboration incredibly difficult.

How did Databricks solve Expedia's ML fragmentation?

This video demonstrates how Expedia built a unified ML platform centered on the Databricks Data and AI platform. This includes centralized data storage, a model registry for managing models and metadata, standardized deployment services, and an ML workbench for team collaboration.

How do I move a machine learning model from experimentation to production?

This video demonstrates Expedia's approach: centralize model development using Databricks notebooks with CI/CD pipeline integration, then use model deployment services to standardize the handoff from data scientists to ML engineers. The unified ML platform on Databricks automated the ML lifecycle from prototyping through production deployment, reducing the cycle from months to days and enabling multiple iterations per year.

Should I build machine learning on my data platform or use a separate ML platform?

This video shows why Expedia chose to build a unified ML platform anchored on Databricks Data and AI platform rather than maintaining separate systems. A single platform provides centralized data storage, model repository with metadata management, standardized deployment workflows, and an ML workbench for collaboration. This approach eliminated 10 different silos, reduced duplicate efforts, and accelerated model deployment 4x faster than their previous architecture.

What should we shortlist for production machine learning at enterprise scale?

This video reveals Expedia's production ML platform now runs 350 active models serving 900 billion predictions annually with reliability and scale. Key features include unified infrastructure supporting both real-time and batch models, centralized governance through Unity Catalog, standardized deployment practices, and collaborative tools for 70% more ML practitioners. The video demonstrates that enterprise ML platforms must handle scale, governance, and team collaboration simultaneously.

Full transcript

Welcome to this session. I'm from Expedia. I lead the data and AI function there. I'm really excited to be here. Now imagine a world that travel experience from planning your dream vacation to solving unexpect unexpected interruption of your trip is all powered by AI intuit intuitively instantly and seamlessly. Right. At Expedia Group, we are turning that vision into reality, accelerating AI to power travel experience for millions of travelers and partners worldwide.
And to get there, it's not just about building bigger, fancier models. It's also about building interoperable AI system that can scale and evolve and deliver business impact at scale. So many years ago we faced a lot of challenges. One of the challenge was at Expedia group we had 10 different ML system and they were all operating in silos. Every team have their own data break instance and there wasn't a single system that can handle the ML full life cycle from training to deploy deployment your model uh end to end and our model metadata was very scattered all over the place.
Teams often rebuild models and not able to leverage the works existing work like they were not able to leverage each other's features or data pipelines and the infrastructure that there wasn't standard deployment practice as well making scalings and collaboration incredibly difficult. Fragmentation not only create operational inefficiency, duplicate efforts, slow down our ability to innovate. It became very clear to scale AI for the future of travel.
We have to scale ourself. So what is the solution? The solution here is we got rid of the siloed landscape. We build a unified ML platform anchored on top of data brick. Let me break apart the component for you. So the center of it is is built on top of a unified data brick instance that supporting both model development as well as production. We created a model repository services that centralized the model uh storage and with metadata discoverabilities one place for all our model information.
We shipped a thing called model deployment services that simplified and standard model deployment workflow. Uh we also built something called the ML workbench that allow all the ML practitioners within Xedia whether it's machine learning scientists or machine learning engineer or even analyst to collaborate and use the uh and build on top of each other's work. So this integrated unified ML platform is also uh embedded with the broader EG ecosystem and thanks to data brick open principle it's integrated with our own EG data lake our streaming platform and our in-house experimentation platform.
So what are the results? So we transformed now we have a centralized scalable and gardenable ML platform. Today about 350 active models are running on top of this platform. Some are real time, some are batch models. They are used to power personalization experience. They're also helping our partners to grow. For example, we have market forecasting models so our suppliers can take advantage of that future market supply and demand information to maximize their yields and revenues and about 900 billion predictions are running annually on top of this ML platform.
Imagine the availability and scale and because of those transformation our speed to innovate is four times faster. So our model is now having more um velocity running AB testing velocity and throughput four times faster than two years ago. The best part is 70% more machine learning practitioners, scientists, engineers are collaborating with each other on top of this single centralized ML hub. So along the way, we also achieved the three big unlocks using uh AI interoperability.
Let me walk you through each of them. The first big unlock of AI interoperability with data brick is accelerating our AI development through centralized collaborative workflow. Our ML work development life cycle previously was very slow from testing to live AB testing took months. So that means we were only able to iterate maybe twice or three times a year. That's not enough. And then the ML teams were operating in silo and reinventing the views.
I'm talking about the model backbones, the model architecture, features, data pipelines, all reinventing those views which each development. So we know we need to work faster, smarter. So we adopt a new approach. In the new approach, we use data brick notebook and that promotes centralized efficiency on exploring experimentation model prototyping and not only that we integrate that notebook with our internal CI/CD pipeline.
So the handoff between machine learning scientists to machine learning engineer become seamless. Right? So the entire ML life cycle from prototyping to deployment in production become automated and thanks to data brick scalability and elasticity we truly accelerate our experimentation dramatically reducing that cycle from months to days. So now we can iterate many many times a year and teams start to collaborate in real time sharing resources with each other and move faster from prototyping to production.
For example uh within my team I have one works on traditional hotel ranking the other works on vacation ranking. You know previously they were operating in silo. Now they can share the same deep neuronet network backbone common similar features and pipelines. The second big unlock from AI interoperability is secure and governance on top of our 70 pabytes of data. Expedia has been in business for almost three decades.
That means we have huge amount of data both depths and breath of data. Our data power personalized the traveler experience and also help our partner to grow but only if we can use those data efficiently. We had challenge right. So our models and data were in silo. My own machine learning scientists have to navigate different data physical datas pipelines or even the garment rules even before they can build data models or building uh the model themsel that really slow down the innovation and also create a lot of security risk.
So we need to unify our data practice and data governance. We start to use Unity data catalog from data brick. It's now seamlessly integrated with our vast data assets and grand access to those assets. The workflow also became simplified making it easier for data scientists to use access those structured nonstructured data sensitive nonsensitive data and also unity catalog not just manage those data assets it also manage access to the models and all the files around the models due to the open architecture principle of data brick the ML workflow ML workload also integrate with our internal data lake streaming platform and AB testing platform.
The third last big unlock from AI interoperability is a much more scalable and reliable production environment for bigger complex data job. Earlier I me I I mentioned we have 350 active models running 900 billion inference or predictions. A lot of those are uh batch jobs. So we're running hundreds very large complex spark jobs daily using data brick. Let me give you an example. We know travelers love reading reviews be before making a purchase.
So on expedia across our expedia. com, hotels. com and vacation rental we have millions of reviews. So we use data brick to process those reviews uh to generate very concise concise and relevant summaries so that it's on the fingertip of the travelers to make decisions quickly and those review models uh summarization models are using NLPs and state-of-the-art geni models. There's also performance optimization. We use data brick photon that help us to cut query query runtime as well as saving cloud cost on average by 20% for some cases.
You know as an engineer now you don't have to trade off. You can benefit from both better speed and saving. All this foundational work allow Xedia to scale GNI applications across our business without the need of major replplatforming or re-engineering. Also we are leveraging data brick to fine-tune opensource large language model. So it opens new doors for Expedia to leverage cuttingedge genai applications. I'm going to switch gears.
Talk about responsible AI. As we're using more AI to drive decisions, it becoming increasingly critical to build trust through responsible and explainable AI not just with our external external stakeholders like traveler and partner but also with our internal leadership team legals and responsible AI committee because if we couldn't understand how those AI make decisions Why making those uh predictions? How could we know it's living and bracing our values to our uh travelers?
So the principle of our responsible AI committee has four pillars transparency, fairness, accountability and privacy. And those are not just buzzwords. They are actually living and breathing cornerstones how we operate, how our system operate. So we build the trust into the system, the process as well as the mindset. We integrate ML flow into our model registration services and that give team the ability to monitor and track things like mo data drift.
So our team instead of being reactive to data drift or other issues, now they can proactively identify and addressing those issues. So monitoring and explaining model behavior is more in real time and governance and compliance are built into the system not balded on later. So what's the results? So Xedia can run scalable AI more responsibly. Our model is more reliable, explainable and transparent to our product managers or external stakeholders.
And we are meeting standards with speed not friction. Right? So I talked a lot about the technical sides. Let's also look at some business impacts. The three technical unlocks we achieved with data break also having huge business impact. We're iterating 30% faster now on our ML. That means we're shipping models to the market or new features to the market faster. We are operating with lower operation and infrastructure cost increase our overall company's efficiency and competitiveness.
We're driving richer and more personalized immerse experience for our traveler and partners. Let me share another example. So we do a lot of recommendations on our side and within the app that's all drawn on our reach inside on our data. For example, we can quickly uh predict and extrapolate whether this is a romantic getaway intention or business trip or family trip and based on those intentions will recommend different destinations or hotels and properties.
At Expedia, AI interoperability is no longer just a technical achievement. It also unlock uh it's also a strategic business multiplier. Unlock real value for the company and our partner. Let's also looking ahead. So, Expedia has even bigger ambitions to accelerate AI to enhance travel experience for our millions of traveler and partners. Data Break continue to be part of that journey. We're using data brick advanced compute capability to power next generation Gen AI use cases.
We are working on deeper personalization, smarter predictions and more immersed experience for traveler and partner. Let me share two very near uh real examples. So hotels. com travelers now can explore, shop and manage their inrip whether is a cancellation or changing dates all with one single AI assistant. Previously they have to go to two different places. And if you are a vacation host, vacation uh rental host, you can accept addit or regenerate a SEO friendly property description uh using our GNAI capability.
That means your on boarding is faster and your listing is higher quality and more attractive. So you can see we're exploring more sophisticated GNAI applications with a focus on flexibility, innovation, and the traveler delight. It's really pushing the boundary of what AIdriven experience can achieve. This is my last slice. So what are some of the takeaways? You can see we are ex establishing AI interoperability with data brick and it's not just about technology it's also about faster innovation stronger collaboration and delivering tangible bigger business impact to achieve this we did a few things we unified our ML development system get rid of the fragmentation we unlocked the interoperability across our vast data ecosystem.
And thanks to our machine learning engineers, we accelerated AI innovations through seamless workflow and rapid prototyping. And last but not least, we build reliable and scalable foundation that deliver high quality AI solutions. Data Break continue to be a key partner in our journey for the future and thank you all of you and we're building the future of travel. Thank you very much.

Learn more about the Databricks Data and AI platform.