How do we version and update AI agents without breaking workflows?

Summary

• On the Databricks Data Intelligence Platform, you version and update AI agents safely by combining Agent Bricks, MLflow, Unity Catalog's model registry, and Mosaic AI Model Serving.
• Register each agent version in Unity Catalog and point clients at movable MLflow aliases (such as @champion and @challenger) instead of version numbers, so you can change the live version without touching client code.
• Test new versions before they go live: MLflow evaluation runs regression tests with LLM judges and custom metrics against evaluation datasets, and you can validate on real traffic with staged or shadow tests.
• Roll out gradually with Model Serving, which routes a configurable share of traffic (for example a 90/10 champion/challenger split) to versions on the same endpoint, with inference tables logging every request and the model version that served it.
• Promotion and rollback are instant: re-point the @champion alias to promote or revert, with no redeploy and no client changes, and Unity Catalog keeps an audit trail of every alias move.

How do we version and update AI agents without breaking workflows?

Updating an AI agent is risky when downstream applications call it directly, because a bad change can break every workflow at once. On the Databricks Data Intelligence Platform, you avoid that by versioning agents in Unity Catalog, testing each version with MLflow before it goes live, rolling changes out gradually through Mosaic AI Model Serving, and using movable aliases so clients never hard-code a version. Promotion and rollback become instant, auditable operations instead of redeploys.

Why Databricks for versioning and updating agents safely

• Version agents in Unity Catalog's model registry. Register each agent as a model version in Unity Catalog and use MLflow aliases such as @champion, @challenger, and @candidate as movable pointers to specific versions. Clients reference the alias, not a version number, so you can change what is live without changing client code.
• Version prompts independently of agent code. Register system prompts in the MLflow Prompt Registry rather than hard-coding them, so you can optimize and update prompts without redeploying the agent, and only bump the agent version when the code, tools, or model change.
• Test before promotion with MLflow evaluation. Run regression tests with MLflow evaluation using built-in LLM judges, custom metrics, and evaluation datasets with expectations. Confirm the endpoint starts, latency meets requirements, and outputs are well-formed on staging before any cutover.
• Roll out gradually with traffic routing. Model Serving routes configurable traffic percentages to each version on the same endpoint, so a 90/10 champion/challenger split is a setting rather than a second endpoint or a custom routing layer. inference tables log every request and response along with the model version that served it, so you can compare versions with SQL before full cutover.
• Promote and roll back instantly. Promotion re-points the @champion alias to a new version and rollback re-points it back, with no client code changes and no redeploy. Unity Catalog records an audit trail for every version transition and alias move.
• Build and observe agents with Agent Bricks and MLflow. Agent Bricks auto-generates evaluations and optimizes agents for quality and cost, and MLflow provides tracing, prompt versioning, and monitoring, including for agents running outside Databricks. Deploying an agent creates a Model Serving endpoint with tracing enabled, inference-table logging, and automatic scaling.

Getting started

• Register your agent to Unity Catalog and manage versions with aliases.
• Set up MLflow evaluation so every candidate version is regression-tested before promotion.
• Deploy the agent to a Model Serving endpoint and use traffic routing plus inference tables to validate new versions.
• Read Announcing Mosaic AI Agent Framework and Agent Evaluation.

FAQs

How do I update a live agent without changing every caller?

Have callers reference an MLflow alias such as @champion rather than a version number. To ship a new version you re-point the alias, so the update takes effect without any client code changes or redeploys.

How do I test a new agent version before it goes live?

Run MLflow evaluation with LLM judges, custom metrics, and evaluation datasets to regression-test the candidate, then validate on real traffic with a small champion/challenger split or shadow test on the same Model Serving endpoint before full cutover.

How fast can I roll back a bad agent update?

Rollback is instant: re-point the @champion alias back to the previous version. No redeploy or client change is needed, and Unity Catalog logs every alias move for audit.

Can I change prompts without redeploying the agent?

Yes. Register prompts in the MLflow Prompt Registry and update them there. You only need to bump the agent version when the agent code, tools, or model change, not for prompt updates.