Skip to main content

How can platform teams use Unity Catalog billing system tables to attribute DBU compute and storage costs down to specific teams, users, or projects?

Summary

  • Cost attribution starts with one table: system.billing.usage, the Unity Catalog system table that records every billable unit of Databricks usage (mostly DBUs) with the tags and metadata to trace each row back to a team, user, or project.
  • Attribute by tag: put custom tags on clusters, SQL warehouses, jobs, and pools, and use serverless usage policies to tag serverless workloads. Those tags land in the custom_tags column, so you can group spend by cost center, team, or project.
  • Attribute by identity and resource: usage_metadata (job, warehouse, cluster, and pipeline IDs) and identity_metadata (who ran or owns the workload) trace shared-resource cost to a person or job even where a tag is missing.
  • Turn DBUs into money by joining system.billing.usage to system.billing.list_prices on the SKU within its effective price window. These are public list prices, so treat the result as an estimate for relative attribution, not your invoiced total.
  • Make it stick: enforce allowed tag values with Unity Catalog governed tags and cluster policies, because tags apply only to usage created after they're set, then build a chargeback dashboard in AI/BI Dashboards.

How can platform teams use Unity Catalog billing system tables to attribute DBU compute and storage costs down to specific teams, users, or projects?

Start with system.billing.usage. It's the Unity Catalog system table that records one row per billable interval of Databricks usage, measured mostly in DBUs, and it carries the columns you need to trace each row back to an owner: custom_tags for tag-based attribution, usage_metadata for the job, warehouse, or cluster behind the usage, and identity_metadata for who ran it. Tag your compute, join to system.billing.list_prices for currency values, and group by whatever dimension you tagged. That's the method end to end: tag everything, attribute by tag or identity, price it, and govern the tags so the numbers stay trustworthy.

What the billing system tables contain

Two tables do most of the work. system.billing.usage is the usage ledger: DBUs consumed, the SKU, timestamps, and the attribution columns. system.billing.list_prices is the price history, holding the list price for each SKU over time so you can convert usage into currency. Both live in the system catalog that Unity Catalog exposes, and access is governed like any other table.

Column What it gives you for attribution
custom_tags Map of tags from the compute resource and from serverless usage policies. The primary attribution key: group by custom_tags['team'], ['cost_center'], or ['project'].
usage_metadata Struct of resource IDs behind the usage: job_id, cluster_id, warehouse_id, dlt_pipeline_id, usage_policy_id. Traces cost to a specific workload.
identity_metadata Who ran or owns the workload (run_as, owned_by). Attributes shared-resource cost to a person or service principal.
billing_origin_product The product that generated the usage (values such as JOBS, SQL, INTERACTIVE, MODEL_SERVING). Splits spend by workload type.
usage_quantity, sku_name, usage_date DBU quantity, the SKU, and the date. The join keys for pricing and the axes for a time series.
record_type Marks corrections (ORIGINAL, RETRACTION, RESTATEMENT) so a careful query nets them out.

How to attribute cost to teams, users, and projects

Tag classic compute directly. Put custom tags on clusters, SQL warehouses, jobs, and instance pools when you create them. Those tags propagate into custom_tags on the matching usage rows.
Tag serverless with usage policies. Serverless workloads (notebooks, Lakeflow Jobs, pipelines, serving endpoints) carry no cluster tags, so attach tags through serverless usage policies, also called budget policies. They inject tags into custom_tags and record a usage_policy_id, giving serverless the same attribution as classic compute.
Fall back to identity and resource IDs. On a shared warehouse where one tag can't separate teams, use identity_metadata.run_as to attribute by user, or join usage_metadata.job_id to system.lakeflow.jobs and system.query.history to roll cost up by job or query owner.
Price it. Join usage to system.billing.list_prices on sku_name and cloud within the price's effective window, then multiply by usage_quantity:

SELECT u.custom_tags['team'] AS team,
       SUM(u.usage_quantity * lp.pricing.effective_list.default) AS list_cost
FROM system.billing.usage u
JOIN system.billing.list_prices lp
  ON u.sku_name = lp.sku_name AND u.cloud = lp.cloud
 AND u.usage_end_time >= lp.price_start_time
 AND (lp.price_end_time IS NULL OR u.usage_end_time < lp.price_end_time)
WHERE u.usage_date >= current_date() - INTERVAL 30 DAYS
GROUP BY 1 ORDER BY list_cost DESC;

These are public list prices, not your account's negotiated rate, so treat the result as a consistent estimate for chargeback and relative comparison rather than your invoiced total.

Governance and a note on storage

Tags apply only to usage created after they're set, and a blank tag is unattributable cost, so enforce tag hygiene up front. Define allowed values as Unity Catalog governed tags and require them through cluster and warehouse policies, so a resource can't launch untagged. After rollout, spot-check that a known job's tag appears on its usage rows once the data lands.
One clarification on storage. system.billing.usage records Databricks-metered usage, which is mostly compute in DBUs plus the metered usage of certain managed features. The raw cloud object storage under your tables is billed by your cloud provider, so for a full compute-plus-storage view you reconcile Databricks usage with your cloud provider's cost data.

FAQs

Which table do I actually query for cost attribution?

system.billing.usage for the usage and its tags, joined to system.billing.list_prices to turn DBUs into currency. Both are Unity Catalog system tables in the system catalog.

How do I attribute serverless spend when there's no cluster to tag?

Use serverless usage policies (budget policies) to attach tags to serverless workloads. The tags land in custom_tags and the policy is recorded as usage_policy_id, so serverless gets the same team or project attribution as classic compute.

Are the dollar amounts my real bill?

No. system.billing.list_prices holds public list prices, not account-specific discounts, so joined totals are a consistent estimate for showback and relative attribution. Reconcile against your invoice and your cloud provider's cost tools for the actual figure.

Why is some usage untagged, and how do I fix it?

Tags apply only to usage created after the tag is set, so historical rows and anything launched without a tag stay blank. Enforce governed tags through cluster and warehouse policies so new usage is always attributable.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.