Unity Catalog audit logs and system tables explained

Summary

• Audit logs in Unity Catalog record every data access, modification, and governance action, creating an immutable record for compliance and forensics.
• System tables expose audit logs as queryable Delta tables, enabling SQL-based analytics on data governance itself.
• Audit logs support regulatory requirements (SOX, GDPR, HIPAA) by tracking who accessed what data, when, and from where.
• System tables answer governance questions: which users accessed sensitive data, which tables lack lineage, which catalogs have over-permissioned users.
• Access to audit logs follows role-based access control, with 365-day retention by default.

Unity Catalog audit logs and system tables explained
Unity Catalog audit logs capture every action in the system: SQL queries that read or modify data, permission grants and revokes, table schema changes, and who performed each action and when. These logs are immutable and queryable through system tables, meaning auditors and engineers can reconstruct exactly what happened to data at any point. When compliance requires proving that sensitive customer data was accessed only by authorized personnel, audit logs are the source of truth.
System tables expose these logs as queryable Delta tables within the Unity Catalog itself. Rather than exporting logs to a separate SIEM or compliance tool, data engineers query audit logs directly using SQL, combining them with table lineage, schema history, and permission records to understand governance across the entire system.

What are audit logs and system tables?

Audit logs are immutable records of actions on data and metadata:

• Data access: SELECT queries that read tables, volumes, or models.
• Data modification: INSERT, UPDATE, DELETE, TRUNCATE operations; schema changes; table renames.
• Governance actions: grant/revoke permissions, create/delete roles, object creation.
• User activity: login events, failed authentication attempts, workspace API calls.

Each log entry includes timestamp, acting principal (user or service principal), action type, resource affected, and success/failure status.
System tables expose audit logs and metadata:

• system.access.audit_logs: Primary audit table with one row per auditable action.
• system.information_schema.catalogs, schemas, tables, volumes: Object metadata.
• system.information_schema.table_lineage: Dependencies between tables.
• system.information_schema.column_lineage: Which source columns feed into target columns.
• system.information_schema.grants: Current permission assignments.

System tables are read-only and automatically updated; they reflect current system state.

Key capabilities to evaluate

Capability	Why it matters
Immutable audit trail	Prevents tampering; satisfies compliance audit requirements
Complete action coverage	Every data access and governance change is logged with no gaps
Queryable with SQL	Auditors use standard SQL without export tools
Near-real-time availability	Logs appear within minutes, enabling rapid investigation
Fine-grained detail	Includes query text, returned row counts, and affected columns
Lineage tracking	Understand which upstream data feeds into reports and models
Column-level governance	Track which specific columns were accessed
Multi-workspace audit	Aggregate audit logs from multiple workspaces

How Databricks approaches audit logs

Unity Catalog automatically logs all access and governance events into system.access.audit_logs. Each row includes user identity, action type (SELECT, INSERT, GRANT, etc.), resource type (TABLE, VOLUME, CATALOG), object identifier, timestamp, and status.
Data engineers query system tables to audit data access. Finding all users who accessed a specific table in the last 30 days becomes a simple SELECT statement. Combining audit logs with schema metadata lets you find all tables matching patterns (like sensitive tables containing personal data) and see who accessed them.
Lineage system tables show which tables depend on sensitive data sources, enabling impact analysis when access controls change. Permission system tables list all grants, revealing over-permissioned roles or catalogs with excessive access.

Use cases

Regulatory compliance: SOX, GDPR, and HIPAA audits require proving that access controls work. Query system tables to generate reports on who accessed sensitive schemas and when permissions changed.
Incident investigation: When data is deleted or a query behaves unexpectedly, query audit logs to find which user or job made the change, what was executed, and whether it succeeded.
Usage analytics: Identify frequently-used tables, dormant schemas, and which columns matter most. This informs governance priorities and cost optimization.

FAQs

How long are audit logs retained?

Audit logs are retained for 365 days by default. For longer retention, you can archive logs to your own Delta tables using a pipeline.

Who can view audit logs?

Access follows role-based control: workspace admins can query audit logs by default. You can grant SELECT on system.access.audit_logs to auditors or compliance teams with narrower permissions.

Do system tables show query results?

No. System tables include query metadata (which table was accessed, rows returned, timestamp, user) but not the actual result data, balancing compliance needs with privacy.

Can I send audit logs to an external tool?

Yes. Create a pipeline using Lakeflow Pipelines to read from system.access.audit_logs and write to an external SIEM, data warehouse, or security system for centralized compliance dashboards.