How do I monitor every AI interaction across my organization?

Summary

• One governed gateway for all AI traffic. The Databricks Unity AI Gateway gives you centralized observability over every model and MCP request in your organization, backed by Unity Catalog system tables. See The AI Gateway: a governance layer for agentic AI.
• Capture every request and response. Inference tables log complete request and response payloads, latency, status codes, and errors to Delta tables in Unity Catalog, giving you a queryable, auditable record of exactly what was sent and returned.
• Track usage and real cost in system tables. The system.ai_gateway.usage system table records token counts, latency, requester identity, and request tags, with actual dollar costs calculated automatically.
• Attribute and control spend. Attribute cost by endpoint tag, request tag, identity, model, and provider, and set AI budgets that alert immediately when a threshold is exceeded.
• See it all in the Governance Hub. Pre-built dashboards track token usage, governed traffic, per-user spend, active models and users, and guardrail coverage across your estate.

How do I monitor every AI interaction across my organization?

On Databricks, you monitor every AI interaction through the Unity AI Gateway, a single governance layer that routes model and MCP (Model Context Protocol) requests and records them in Unity Catalog. Because the gateway sits in front of all governed AI traffic, it produces one centralized, queryable, auditable record of who called what, what was sent and returned, how long it took, and what it cost — visible in system tables, endpoint metrics, and the Governance Hub. See The AI Gateway: a governance layer for agentic AI.

Why Databricks Unity AI Gateway gives you full visibility

• Centralized usage tracking. The Unity AI Gateway usage dashboard tracks request volume, token consumption, and latency for all endpoints governed through the gateway, and each governed model or MCP service has a built-in Metrics tab for health monitoring in Catalog Explorer. See Monitor usage and performance.
• Payload and inference logging. Inference tables capture every request and response across model calls and MCP interactions, storing complete request and response payloads, latency, status codes, and errors as Delta tables in Unity Catalog. This lets you debug by tracing exactly what prompt was sent, what the model returned, and where a call broke. See Capture every interaction with payload logging and Unity AI Gateway inference tables.
• System tables for usage and cost. All requests log to Unity Catalog system tables with actual dollar costs, not only token counts. The system.ai_gateway.usage table captures token counts, latency, requester identity, and request tags, and automatically calculates cost across provisioned throughput, pay-per-token usage, and external model pricing — so you can query organization-wide AI usage directly with SQL. See Monitor usage, cost, and risk.
• Rate limits and AI budgets. AI-tailored budgets let you set spending limits for your organization's AI workloads, and when a user crosses a threshold it is surfaced immediately. See Introducing AI spend controls in the Unity AI Gateway and cost observability.
• Per-model and per-user attribution. You can attribute cost across multiple dimensions: endpoint tags to group by team, environment, or cost center; request tags for dynamic attribution; identity to aggregate spend by user or service principal; and by model and provider to see what is driving cost. See Catch AI cost surprises early.
• Audit logging. Every request logs the requesting identity and timestamp, and for MCP calls the connection name, HTTP method, and whether the call was made on behalf of a user. Unity Catalog permissions control who can see this data, and inference tables maintain complete audit trails for compliance. See New updates to bring security and governance to GenAI models.
• Dashboards in the Governance Hub. The AI vertical in the Governance Hub provides pre-built dashboards and tiles for token usage, governed traffic, AI budgets, AI usage and spend over time, active models and active users over the trailing 30 days, per-user spend, and guardrail coverage. See Governance Hub for AI.

Getting started

• Read The AI Gateway: a governance layer for agentic AI for the full observability model.
• Route your model and MCP endpoints through the Unity AI Gateway and enable inference tables for payload logging.
• Query the system.ai_gateway.usage system table and use the usage, cost, and risk dashboards.
• Set AI budgets and track spend in the Governance Hub.

FAQs

How does the Unity AI Gateway record every AI request?

The gateway routes model and MCP requests and logs them to Unity Catalog. Inference tables capture complete request and response payloads, latency, status codes, and errors as Delta tables, creating a centralized, queryable record.

Which system table holds AI usage and cost data?

The system.ai_gateway.usage system table records token counts, latency, requester identity, and request tags, with actual dollar costs calculated automatically across provisioned throughput, pay-per-token, and external model pricing.

How do I attribute AI cost to teams and users?

Use endpoint tags to group by team, environment, or cost center; request tags for dynamic attribution; identity to aggregate by user or service principal; and model and provider dimensions to see what drives cost.

Where can I see organization-wide AI activity at a glance?

The AI vertical in the Governance Hub provides pre-built dashboards with tiles for token usage, governed traffic, AI budgets, spend over time, active models and users, per-user spend, and guardrail coverage.