Where should I host AI agents that need to take actions on data?

Summary

• Model Serving provides real-time agent endpoints for synchronous requests: a user asks a question, the agent responds immediately with actions taken.
• Lakeflow Jobs orchestrates batch agents for scheduled or event-driven execution: run agents nightly to process new data, or trigger them when files arrive.
• Agents running on Databricks execute within the same environment as your data, so they have direct access to Delta tables, SQL, and data lineage without cross-platform calls.
• Stateless agent architectures scale easily; stateful agents (those maintaining conversation history or persistent memory) require careful session management or state stores.
• Route synchronous agents through Model Serving for consistent latency; batch agents through Lakeflow Jobs for cost efficiency and task orchestration.

Where should I host AI agents that need to take actions on data?
Agents that modify data (write to tables, execute queries, trigger workflows) must run in an environment where they can authenticate securely and access the data platform natively. Hosting decisions depend on whether your agents are synchronous (respond to user requests in real time) or asynchronous (run on a schedule or event trigger).

Hosting Options on Databricks

Model Serving for Real-Time Agents Model Serving provides serverless endpoints for agents that need to respond to user requests synchronously. An agent endpoint accepts a prompt, executes the agent logic (including tool calls and data access), and returns results to the user. Model Serving handles scaling, availability, and request routing automatically. Use this for interactive agents: a data analyst queries an agent for insights, and the agent responds within seconds.
Lakeflow Jobs for Batch and Scheduled Agents Lakeflow Jobs orchestrates agents that run on a schedule or triggered by events. An agent job can run nightly to process new data, or trigger when files arrive in a data lake. Lakeflow provides DAG-based orchestration, error handling, notifications, and retry logic. Use this for agents that perform write-heavy actions (loading data, updating models, reconciling datasets).
Compute Context and Data Access Agents running on Databricks execute in a compute cluster alongside your data. This means agents have direct, low-latency access to Delta tables, SQL queries, and data lineage. There's no cross-platform network calls; the agent reads and writes data in the same environment where it computes.

Architecture Patterns

Pattern	Hosting	When to Use
Real-Time Agent	Model Serving	User requests; interactive Q&A; sub-second latency required
Batch Processing Agent	Lakeflow Jobs	Scheduled; heavy writes; event-triggered; cost-sensitive
Hybrid	Model Serving + Lakeflow Jobs	Interactive queries routed to real-time endpoint; bulk work delegated to batch jobs
Streaming Agent	Structured Streaming (on compute)	Continuous data processing; respond to stream events in real time

State Management

Agents that maintain conversation history or persistent state require explicit state management. Stateless agents are simpler to scale.
Stateless Agents Answer each request independently. No conversation memory between requests. These scale trivially: spin up more endpoint replicas, distribute load. Ideal for agents that retrieve and summarize data without needing context.
Stateful Agents Maintain conversation history, multi-turn interactions, or persistent memory (e.g., "remember what we discussed last session"). You must store state explicitly: in a database, a session store, or a distributed cache. Model Serving can host stateful agents if you manage session affinity (sticky sessions route a user's requests to the same replica). Lakeflow Jobs are inherently stateful within a job run but start fresh each execution.

Deployment Considerations

Authentication and Permissions Agents need credentials to access data sources. If an agent runs under a service principal, that principal must have permissions to all data the agent accesses. If the agent represents a user (e.g., runs as the logged-in user), pass the user's credentials securely to the compute environment.
Latency Requirements Real-time agents through Model Serving target sub-second to few-second latencies. Batch agents through Lakeflow Jobs have no strict latency target; they optimize for cost and throughput.
Cost Model Serving charges for provisioned endpoints (always-on) or serverless compute (pay per request). Lakeflow Jobs charges for the compute time the job uses. For bursty workloads, serverless Model Serving is cheaper; for predictable recurring work, provisioned endpoints may be more economical.
Scalability Model Serving auto-scales replicas as traffic increases (for serverless endpoints). Lakeflow Jobs don't auto-scale within a single job run but can parallelize tasks across multiple machines if configured.

How to Deploy an Agent on Databricks

If you're hosting a synchronous agent, package it as a Python function or Docker image. Deploy through Model Serving. The endpoint is immediately available for requests.
If you're hosting a batch agent, create a Lakeflow Job with tasks running your agent code (as notebooks or Python scripts). Define triggers: schedule it to run nightly, or set an event-based trigger (new files in a folder). The job orchestrates execution, handles failures, and logs results.
Both Model Serving and Lakeflow Jobs integrate with MLflow Tracing, so you log all agent actions: prompts, tool calls, data accessed, outputs. This gives you full observability.

FAQs

Can my agent call external APIs from Databricks?

Yes. Use outbound webhooks or registered custom functions. The agent can invoke external APIs as tool calls, just as it calls internal Databricks functions. Ensure network connectivity (VPC/firewall rules) allows outbound traffic.

What if my agent needs to run continuously, not just on a schedule?

Use Structured Streaming or a long-running job. For example, set a Lakeflow Job to trigger whenever new data arrives in a stream, then run your agent continuously via a loop inside the job task.

Should I version my agent code?

Yes. Use Git or Databricks Repos to version agent notebooks and Python scripts. When deploying to Model Serving or Lakeflow Jobs, specify the exact version you're deploying. This enables rollback if an agent update causes issues.