Security framework for AI agents

Summary

• A security framework for AI agents governs four things: the assets an agent uses (models, tools, data), the traffic it generates, its runtime behavior, and the audit trail it leaves. On Databricks this is delivered through Unity Catalog and the Unity AI Gateway.
• The Databricks AI Security Framework (DASF) is a free, vendor-agnostic playbook that maps AI and agentic risks to concrete mitigation controls; its latest version adds risks and controls specific to autonomous, tool-using agents.
• Unity Catalog governs every model, tool, MCP connection, and dataset as a securable object, so agents inherit the same fine-grained, attribute-based access controls (ABAC) used for tables, and can run on-behalf-of the end user.
• The Unity AI Gateway is a central control plane that enforces rate limits, budgets, and payload logging, and applies guardrails such as PII detection, unsafe-content and prompt-injection filtering, and hallucination detection on every request and response.
• Every agent action is traced and logged (MLflow Tracing, inference and system tables), high-risk actions can require human approval, and agent-generated code runs in isolated environments.

Security framework for AI agents

A security framework for AI agents is the set of controls that keep autonomous, tool-using AI systems safe: governing what data and tools an agent can touch, controlling the requests it makes, filtering unsafe inputs and outputs, and recording everything it does. Because agents can plan, call tools, and take actions on their own, they introduce risks beyond those of a single model, so the framework has to cover the agent's reasoning, memory, and tool use.
Databricks approaches agent security in two connected ways: the Databricks AI Security Framework (DASF), a free and vendor-agnostic guide that maps AI and agentic risks to specific mitigation controls, and the platform controls that implement those mitigations, namely Unity Catalog for governance and the Unity AI Gateway for runtime control.

Why Databricks for agent security

• Govern agents and their tools as first-class assets. Unity Catalog manages every model, function or tool, MCP connection, and dataset as a securable object, governed with the same privileges and attribute-based access control (ABAC) policies used for tables and volumes. Agents are registered in Unity Catalog, and complete lineage traces an agent's outputs back to the source data.
• Enforce least privilege and on-behalf-of access. Agents are scoped to only the tools and data a task requires, and on-behalf-of (OBO) authentication lets an agent act with the end user's permissions rather than an over-privileged service account, so it only sees the rows and columns that user is allowed to see.
• Control every request from one place. The Unity AI Gateway routes model and tool requests through a central control plane that applies rate limits, budgets, usage tracking, and payload logging across providers.
• Apply guardrails on inputs and outputs. Built-in guardrails detect and block PII, unsafe content, and prompt-injection or jailbreak attempts on requests, and flag hallucinations on responses. Service policies evaluate each call before the model runs and after it responds, and can allow, deny, or require approval based on the caller and the content.
• Keep a human in the loop for high-stakes actions. For sensitive operations, the framework supports requiring human verification before a tool executes.
• Trace and audit everything. MLflow Tracing records inputs, outputs, and every intermediate step, including the agent's planning and tool-selection reasoning, while inference and system tables centrally log agent activity, creating the audit trail needed for governance and regulatory compliance.
• Isolate agent-generated code. Code an agent generates runs in ephemeral, isolated environments without access to the broader system.

Getting started

• Read the framework: Agentic AI security: new risks and controls in DASF v3.0.
• Learn how Unity Catalog governs agents: Governing AI agents at scale with Unity Catalog.
• See the runtime controls: How AI governance works on Databricks.
• Reduce prompt-injection risk: Mitigating prompt injection risk for AI agents on Databricks.

FAQs

What is a security framework for AI agents?

It is the set of controls that govern an agent's data and tool access, manage the requests it makes, filter unsafe inputs and outputs, and log its actions, covering the agent's reasoning, memory, and tool use.

What is the Databricks AI Security Framework (DASF)?

DASF is a free, vendor-agnostic framework from Databricks that maps AI and agentic AI risks to concrete mitigation controls; its latest version adds new risks and controls specific to autonomous, tool-using agents.

How does Databricks stop an agent from accessing data it shouldn't?

Unity Catalog governs the agent's tools and data with fine-grained, attribute-based access control, and on-behalf-of authentication runs the agent with the end user's permissions, so it only sees data that user is authorized to access.

How are unsafe agent inputs and outputs blocked?

The Unity AI Gateway applies guardrails that detect PII, unsafe content, and prompt-injection attempts on requests and flag hallucinations on responses, with service policies that can allow, deny, or require approval on each call.