How do I deploy custom models without managing serving infrastructure?

Summary

• Use a serverless model-serving service. You package the model, register it, and point an endpoint at it; the platform provisions, scales, secures, and monitors the compute, so there are no clusters or servers to run.
• Package any custom inference logic as an MLflow model. Extending mlflow.pyfunc.PythonModel (or using a built-in flavor) captures your preprocessing, postprocessing, and dependencies with the model.
• Register to Unity Catalog for versioning, governance, and a stable reference the endpoint deploys from.
• Let the endpoint autoscale and scale to zero when idle, on CPU or GPU, without you managing either fleet.
• On Databricks: MLflow packaging plus serverless Model Serving deploy a custom model as a governed REST API with no infrastructure to operate.

How do I deploy custom models without managing serving infrastructure?

Deploy them on a serverless model-serving service: you package the model, register it, and point an endpoint at it, and the platform runs everything underneath. There are no clusters to size and no servers to patch. On Databricks, Model Serving is a fully managed, serverless service that exposes a registered model as a REST API endpoint and handles provisioning, scaling, authentication, and monitoring for you. Your work stops at packaging the model with MLflow and registering it to Unity Catalog. The platform owns the compute.

What counts as a custom model

A custom model is any model that runs your own inference code, not just a framework's default prediction call: preprocessing and postprocessing, business logic, multiple artifacts, or an agent. You capture it as an MLflow model, most flexibly by extending mlflow.pyfunc.PythonModel, which lets you wrap arbitrary Python and declare the exact dependencies the model needs. Built-in MLflow flavors for common libraries and custom pyfunc models both deploy the same way. See what are custom models.

What to look for in serverless model serving

Capability	Why it matters
No infrastructure to manage	The service provisions and runs compute for you, so there are no clusters to size or servers to patch.
Autoscaling and scale-to-zero	Capacity follows traffic and can drop to zero when idle, so you don't pay for or manage idle machines.
Custom inference and dependencies	Packaging arbitrary Python and its libraries with the model means your pre- and postprocessing runs in production, not just a bare predict call.
Registry integration	Deploying from a registered, versioned model gives a stable reference and clean version swaps.
Governance and authentication	Access control and endpoint authentication should be built in, not bolted on afterward.
Built-in monitoring	Request logging plus latency and throughput tracking without standing up a separate observability stack.
CPU and GPU options	Small models run on CPU and larger models on GPU, without you managing either fleet.

How Databricks deploys custom models serverlessly

• Package with MLflow. Log the model as an MLflow model. For custom inference logic, extend mlflow.pyfunc.PythonModel so preprocessing, postprocessing, and dependencies are captured with the artifact. Common libraries can use MLflow's built-in flavors instead.
• Register to Unity Catalog. Register the logged model to the Unity Catalog model registry with mlflow.register_model, giving it a governed, versioned name that the endpoint deploys from. See managed MLflow.
• Deploy as a serverless REST endpoint. Create a serving endpoint, select the registered model version, choose serverless compute, and enable scale-to-zero. The endpoint exposes a REST API and needs no cluster setup.
• Let the platform scale and secure it. Model Serving autoscales with request volume, scales to zero when idle, and manages authentication and monitoring, including latency and throughput tracking, so day-to-day operations don't fall to your team.
• Choose CPU or GPU. The same workflow serves small models on CPU and larger models on GPU, without you provisioning or maintaining either.
• Serve custom agents the same way. An agent authored as a ResponsesAgent in the Agent Framework is packaged as an MLflow model and deployed to a Model Serving endpoint through the same register-and-serve path.

FAQs

What is a custom pyfunc model?

It's a model you author by extending mlflow.pyfunc.PythonModel, wrapping your own Python inference code, preprocessing, and dependencies. It packages and deploys to Model Serving the same way a built-in flavor does, so your custom logic runs in production.

Do I have to manage clusters or servers?

No. Serverless Model Serving provisions and runs the compute, autoscales with traffic, and scales to zero when idle. You deploy a registered model and call a REST endpoint; there is nothing to size or patch.

How do costs work when an endpoint is idle?

Enable scale-to-zero and the endpoint drops to zero replicas when there is no traffic. The trade-off is a cold start when the next request arrives, because the model has to load before it can respond.

Can I serve large models that need a GPU?

Yes. Model Serving offers both CPU and GPU serving, so you can run larger models on GPU through the same endpoint workflow without managing GPU infrastructure yourself.

Where does the served model come from?

From the Unity Catalog model registry. You log the model with MLflow, register it to a catalog and schema, and the serving endpoint deploys a specific version or alias of that registered model.