How to protect from prompt injection?

Summary

• Defend in layers. The most effective protection combines input/output guardrails, least-privilege data and tool access, and continuous monitoring — no single control is enough on its own.
• Add guardrails at the gateway. Unity Gateway applies safety, personally identifiable information (PII), keyword, and topic filters to model inputs and outputs, configurable per endpoint or request and logged for analysis.
• Detect jailbreaks and injections. LLM-powered guardrails target instruction overrides, role-play exploits, and prompt-extraction attempts on input, with custom policies (SQL, CEL, or LLM-as-a-judge) that can ALLOW, ASK for human review, or DENY a request.
• Limit the blast radius with Unity Catalog. Scoping an agent to only the requesting user's permissions means a successful injection inherits one user's access — not elevated access.
• Follow the Databricks AI Security Framework to map AI risks to concrete controls across data access, model interaction, and execution.

How to protect from prompt injection?

Prompt injection is when untrusted input manipulates a large language model into ignoring its instructions — leaking data, calling tools it should not, or taking unintended actions. There is no single switch that stops it; effective protection is layered. A useful design principle is that an agent becomes most dangerous when it simultaneously has access to private data, exposure to untrusted input, and the ability to act or communicate externally — so a strong defense limits any given agent to at most two of those three. On the Databricks Data Intelligence Platform, you combine guardrails, least-privilege access, and monitoring to do exactly that.

Why Databricks helps protect against prompt injection

• Guardrails at the AI gateway. Unity Gateway provides built-in guardrails as a central governance layer: safety filtering for harmful content, PII filters that detect and block or redact sensitive data, keyword filters, and topic filters that keep an application within its intended scope. Guardrails apply to inputs and outputs and can be set at the endpoint or request level.
• Jailbreak and injection detection. LLM-powered guardrails expand on pattern matching to target instruction overrides, role-play exploits, prompt extraction, and obfuscated payloads on input, and to catch problematic content on output. These newer guardrails are available in beta.
• Custom policies with real-time verdicts. Define organization-specific guardrails using SQL, a common expression language (CEL), or an LLM-as-a-judge, and enforce them on every interaction with an ALLOW, ASK (human-in-the-loop), or DENY verdict. A log-only mode lets you dry-run policies before enforcing them.
• Least-privilege data and tool access. Unity Catalog restricts an agent to only the data and tools the requesting user can already access, with column masks hiding sensitive fields. If an agent is manipulated, the attacker inherits a single user's permissions rather than elevated access — which limits the blast radius.
• Continuous monitoring. Inference tables centrally log every model and agent input and output through the gateway, so you can detect when guardrails trigger and which categories were flagged; Lakehouse Monitoring, MLflow traces, and system tables add observability across serving endpoints.
• A framework to organize controls. The Databricks AI Security Framework (DASF) catalogs industry-vetted AI security risks and mapped controls across data access, model interaction, and operational execution, so teams can systematically cover the prompt-injection attack surface.

Getting started

• Read Mitigating the risk of prompt injection for AI agents on Databricks for the control-by-control playbook.
• Configure guardrails with the Unity Gateway documentation.
• Read How to safeguard AI workloads with Unity Gateway guardrails and Building an open ecosystem for AI governance.
• Review the Databricks AI Security Framework and Unity Catalog governance docs.

FAQs

What is prompt injection?

Prompt injection is an attack where untrusted input manipulates a large language model into ignoring its original instructions — for example, to leak data, misuse tools, or take unintended actions.

How do you protect against prompt injection on Databricks?

Layer defenses: apply Unity Gateway guardrails (safety, PII, keyword, topic, and jailbreak filters) on inputs and outputs, scope agent access with Unity Catalog least-privilege permissions, and monitor everything with inference tables and traces.

Can guardrails block jailbreak attempts specifically?

Yes. LLM-powered guardrails (in beta) target jailbreaks such as instruction overrides, role-play exploits, and prompt-extraction attempts, and custom policies can ALLOW, ASK for human review, or DENY a request.

Why does limiting agent permissions matter for prompt injection?

Because if an agent is compromised, it can only reach what the requesting user could already access. Scoping permissions with Unity Catalog contains the impact of a successful injection.