MLOps With Databricks

Summary

• MLOps maturity: End-to-end platforms like Databricks provide comprehensive MLOps features—previously requiring custom implementation—reducing complexity.
• CI/CD best practices: Pre-commit hooks, unit tests, PR reviews, and deployment protection ensure code quality and prevent production errors before they occur.
• Production reliability: Structured MLOps processes replace ad-hoc notebook scheduling, delivering reliable model deployments and operational excellence.

MLOps With Databricks

Talk by Maria Vechtomova, MLOps Tech Lead, Marvelous MLOps
Adopting MLOps is getting increasingly important with the rise of AI. A lot of different features are required to do MLOps in large organizations. In the past, you had to implement these features yourself. Luckily, the MLOps space is getting more mature, and end-to-end platforms like Databricks provide most of the features. In this talk, I will walk through the MLOps components and how you can simplify your processes using Databricks.
Databricks Named a Leader in the 2025 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms: https://www.databricks.com/blog/databricks-named-leader-2025-gartner-magic-quadrant-data-science-and-machine-learning
Build and deploy quality AI agent systems: https://www.databricks.com/product/artificial-intelligence
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

FAQs

Why do data scientists need MLOps practices?

Adopting MLOps practices ensures machine learning models are reliable, maintainable, and production-ready. Without MLOps, organizations risk data quality issues, model degradation, and failed deployments that disrupt business processes.

What is the difference between scheduling notebooks and MLOps?

Scheduled notebooks lack version control, code reviews, testing, and deployment safeguards. MLOps uses CI/CD pipelines with pre-commit hooks, unit tests, and approval workflows—catching issues before they reach production and enabling safe, repeatable deployments.

How do I automate model training, testing, approval, deployment, and retraining?

This video explains how Databricks enables full-stack MLOps automation through CI/CD pipelines, version control integration, and orchestration tools. By packaging machine learning code, implementing pre-commit hooks and unit tests, and enforcing approval workflows, organizations can automatically train, validate, and deploy models to acceptance and production environments. Databricks asset bundles simplify the deployment process, allowing teams to deploy entire projects with a single command while maintaining proper environment isolation and protection.

What steps do I need to move a machine learning model from a notebook to a production API?

Moving from notebooks to production requires implementing CI/CD practices, code packaging, and orchestration on the Databricks Data and AI platform. This video outlines the journey from scheduled notebooks to production-grade MLOps by introducing version control, pre-commit hooks, unit and integration tests, and pull request reviews. Databricks provides task-based orchestration that allows you to split complex workflows into steps with different compute requirements and automatic retry capabilities for failed tasks.

How do I version models, code, features, and training data together?

The Databricks Data and AI platform manages versioning across the entire machine learning lifecycle through configuration files, Git integration, and the MLflow model registry. This video demonstrates how packaging your code with dependencies, storing configuration in version-controlled files, and leveraging Delta Lake's ACID properties ensures reproducibility. When code is merged to the main branch, deployment pipelines automatically handle promotion to acceptance and production environments with proper approval gates.

Full transcript

Well, nice to see you all here. I do my best for you to understand what I'm trying to talk about. Um, and I hope if you're connected via earphones, you can hear me well. So, if you're here, I assume you know what the Malops is about, right? But I will remi remind you that is a set of practices that allows to bring machine learning models to production in a reliable and efficient way. But what is production? Production means that the results of your machine learning models are being used in business processes and actually deliver value.
Let me give you an example. Let's say we have a product manager of fulfillment and that product manager asks a data scientist to deliver a demand forecasting model and he starts a P and that P involves a notebook. There is a notebook and data scientist saying well we need to repeat this process every week so let me schedule a notebook and the there are three steps that are in this notebook globally. So there is data prep-processing model training and there are predictions that are computed and in the end it's written to a delta table and these predictions are used by the fulfillment team.
Well is it production? Yes. Is it efficient? Maybe. Is it reliable? No, it's not reliable. And uh what about version control? And what about code quality checks? So there is a feature in data bricks called version history. You can see you know versions of your notebook. Is it enough? Not really because there are much better practices that you can implement like CI/CD pipelines because version control in notebooks doesn't really prevent mistakes from happening.
You will only know afterwards if something went wrong and you can see who did it but that's about it. So in fact you need to have CI pipelines where you have pre-commit hooks. unit test and integration test and you need to have u PR reviews and approval is required to uh proceed with the deployment and there is a CD pipeline that actually does deployment to acceptance in production environment and there is a deployment protection as well you can set it up on GitHub that approval is requirement to actually deploy something uh to a high environment a setup could be similar to that it's a very simplistic setup that we have one main branch and there are feature branches that are started from the main branch and there is a pull request and there are some CI pipeline runs some testing pre-commit hooks and then you will um after approval it's merged to the main branch and when uh the code is merged to the main branch the deployment starts so acceptance in production and only then when there is approval to on production after people reviewed how things are going non acceptance it's actually going to production and it's tagged with the version.
So this example only would work if your environments are protected. So there are two levels of protection that you can implement on data bricks. First of all it's protection when it comes to datab bricks workspaces. uh only developers can only have access to development environments and there are service principle that have access to um respective environment like development SPN has access to development environment acceptance and production have access to acceptance and production and developing team like demand forecasting team can only get access to high environment after they request access to it and someone can improve it.
So it's like a temporary access that you can get to it. Um the next thing that needs to happen is uh the catalog setup because on data bricks you have a um access to workspaces but you also have access to cataloges and in fact um data scientists should not be accessing uh any high environments either. They can have read access but they shouldn't have right access to any environments. So there are quite some setups that you can implement on the schema and catalog level and the this respective accesses this is something that should be implemented.
So only acceptance SPN can write to acceptance and production SPN can write to accept to production environment and demand forecasting team can read from the production cataloges and write to development cataloges. So why it matters? You have full control. The deployment can only happen from um the CD pipeline and there is also quality assurance. Code only goes to production if there are code quality checks and the tests that have passed.
So one of the things that matters when it comes to unit testing is packaging your code. You can't really do unit testing unless your code is packaged. And this is one of the examples how it can be done. It's some dummy code. It's not really important. But what matters is that every code you write can be modularized and put into um into your the package and the package can be uh created. And that's something that is uh one of the big pillars of what is required for proper MLOps.
Um another thing that matters a lot is orchestration. I was describing a big notebook that maybe has thousand size of code. What if it breaks in the middle, right? Then you would have to rerun the whole notebook just to fix your problem. So you need to look at orchestration differently. I explained that there are three main steps. There is a pre-processing, there is model training and uh predictions. And you can split them all in three different tasks.
There can be more of course than that or there can be some conditional tasks but the benefit of it is that you can retry at any step that where your code has been broken and also different tasks may have different requirements in terms of libraries but also in terms of compute like pre-processing can be spark heavy and you need to use uh distributed computing but when you look at the model training it's usually well not always but quite often it's uh basically single compute that you need for that task and using um pi spark or uh clusters that uh are required for pre-processing would not be really efficient for this.
Um another thing that matters for orchestration because you split our tasks. It can be that you need to reuse the package that we created in all these tasks and you also have some um configurations file like a project configuration that has information about the catalog schema you use and maybe some model parameters and features you use to train your model. uh and that's another reason why packaging actually uh makes the whole difference and we just make things a lot more complicated than a notebook right so we introduced a package and we introduced all kind of dependencies and um there is a feature on datab bricks called datab bricks asset bundles that actually make it much easier to deploy all your code with all your dependencies I will not be really go deep into what that uh is doing but it will make your life much easier if you're deploying things on data bricks And it just needs one command datab bricks bundle deploy and the deployment will happen.
Um but what I was talking about isn't it just devops right? Why is it a mlops in the first place? Um mlops is an extension in devops in fact. So there are four main components that um are overlapping for any software engineering system. There is code, there is data model, there is environment and infrastructure. If all these components are the same, the results of a software system are guaranteed to be the same. But it's not like that for a machine learning system.
There is one extra concept, statistical properties of data that matter. And if statistical properties of data have changed, even though the rest hasn't changed, your whole system is broken. That's why you need to have extra tools just to make sure that everything is working as expected. So tracibility and reproducibility is really important for machine learning deployments. So for any machine learning model uh run and deployment you need to be able to look up what code and commit on git was responsible for the deployment infrastructure environment used uh for training and serving data used for model training and also the generated machine learning model artifacts and these components are really required to be able to go back to experiment at any moment of time.
And this is something that can be implemented using MLflow. MLflow provides a lot of uh possibilities to log information related to a run. Um your project can be uh bundled across the one experiment and each run each model training run can have a separate ML flow run. You can log input like log data that belongs to a certain model run log model log metrics and log parameters. And there are other artifacts that you can log as well.
Um, and another thing that we found very useful is adding text to your runs and to your models. And text can commit git commit hash meaning that you know what code was responsible for the run but also the run ID on data bricks. uh because that's something that um is available on MLflow right away but it's a task run ID and you want to have a global run ID of the whole deployment so that you can go back and look at it and when you log a model using MLflow you can then register that in unity catalog and that can be used later for serving or for other uh purposes there is another component uh in ML in data bricks called lake cow monitoring that's something that um also uh helps a lot with tracking the statistical properties of the data and there are two aspects to it.
So when you have uh data uh coming in so I was talking about uh cataloges that are created by the data engineering teams those can uh contain uh customer information transaction information and product information all relevant for demand forecasting projects. You can monitor whether there is something wrong going on with that data and you can set up alerting on that which uh can notify the users and data scientists before actually they start training in the first place and then there can be also a monitoring table set up on the predictions that data scientists generate and this can be done on the data properties but also on the model drift itself if you know the ground truth and that's one of the limitations of pretty much any monitoring system that is out there these days, you must know the ground truth before you actually can say whether there is something going wrong with your model and it can be too late.
Uh so it's important really to track that very well. So I mentioned a lot of tools uh that are there out there on the databicks platform and this is the whole overview of all the tools that are relevant for MLOps. We have an orchestration which is datab bricks workflows. We have experiment tracking. We use a ML flow for that. Compute which is data bricks compute. It can be serverless. It can be standard compute. There are different options for depending on what you need to do.
There is feature engineering. It's a dedicated feature engineering package uh that integrates with unity catalog. You have model registry which is also unity catalog but you use ML flow to register your models. You have serving, model serving, feature serving, model serving with feature lookups, all variations of that. Um you have monitoring, you have observability and vector search which are LLM related features and you have two components which matter a lot.
It's a governance component for Unity catalog where you can basically know everything about going on with your systems and you have glue which is the datab bricks asset bundle which is relevant for anyone doing anything on data bricks pretty much. Um, but didn't we just make it all more complicated, right? We started with a notebook which had 1,000 lines of code and now we have, you know, a package. We have many scripts, we have many arguments and all of that and just why why did we do that, right?
Why? So, I've done it many times in my career. Then that's something that just doesn't necessarily make your life more complicated. The risk of mistakes are significantly reduced. Uh and mistakes can be very expensive. They can worth be worth millions of dollars. time to production. You would think it goes up but it actually goes down because data science teams can be fully responsible for the whole product end to end without relying on other teams like DevOps teams and um you know this relationship between two teams tend to make things a lot more complicated building the wall between different departments and in corporate setup proper MLO's practices can bring millions and that's something that I've done multiple times over my career myself.
Um, so it might be hard to do things the right way from the very beginning, but once you learn to do it right, it will be hard to do it wrong. And uh, that's why I believe everyone should should be adopting MLOps in the right way. So if you want to learn more, we have a course on this topic end to end MLOps with data bricks. Um please uh free to scan the QR code and uh if you're interested I can uh tell you more about it and um I'm also writing a book on Malops with data bricks for O' coming in January 2026.
Uh it has a cute chick on it. It's I don't know why they chose it. I wanted a lion but they have a chick so it's pretty funny. And your early release is already available so you can find it on the platform. Thank you so much. And if you have questions, please ask me. And I can throw a crab at you if you ask me a question.

Learn more about the Databricks Data and AI platform.