Skip to main content

How can I export Lakebase metrics and logs to an OpenTelemetry-compatible observability platform?

Summary

  • Every Lakebase project runs a managed OpenTelemetry Collector that scrapes compute metrics, collects Postgres logs, and forwards both in OTLP format to the observability endpoint you configure.
  • Because the export speaks the OpenTelemetry Protocol, telemetry lands in any OTLP-compatible backend your team already runs, with no custom agent to install on the database.
  • You turn it on from the Lakebase app under Settings, Integrations, OpenTelemetry integration, where you set the protocol, endpoint URL, authentication, and resource attributes.
  • Exported signals cover both host metrics (CPU and memory) and database metrics (connection counts, database size, replication delay, row activity), with Postgres metrics carrying a lakebase_ prefix.
  • For deeper analysis, Advanced Postgres Telemetry writes query plans, wait events, and schema-change history as Delta tables in your own Unity Catalog, and built-in dashboards show recent metrics and active queries out of the box.

How can I export Lakebase metrics and logs to an OpenTelemetry-compatible observability platform?

Point Lakebase at your observability platform's OTLP endpoint and it handles the rest. Every Lakebase project runs a managed OpenTelemetry Collector that scrapes metrics from the compute serving your database, receives the Postgres logs, and forwards both to the OTLP endpoint you configure. Because the data leaves in OpenTelemetry Protocol (OTLP) format, it flows into whatever OTLP-compatible backend your team already uses, so you can monitor the database next to the rest of your stack without installing a custom agent or running your own scraper.

What Lakebase sends over OpenTelemetry

OpenTelemetry is the vendor-neutral standard for collecting metrics, logs, and traces, and OTLP is the wire format its collectors speak. Lakebase exports two kinds of signal. Metrics split into host metrics for the compute, such as CPU and memory, and database metrics such as connection counts, database size, replication delay, and row activity. Database metrics carry a lakebase_ prefix (for example, lakebase_connection_counts) so they are easy to filter, while host metrics keep their standard Prometheus-style names. Logs are the Postgres server logs, forwarded through the same collector. One detail matters for serverless Postgres: metrics and logs come from the active compute, so when scale-to-zero suspends a compute it stops emitting until the next query wakes it.

What to look for in database observability

The point of exporting telemetry is to catch problems early and explain them fast. These are the capabilities that decide whether that is possible:

Capability Why it matters
Standard-protocol export (OTLP) Sending telemetry in an open format lets you use any compatible backend and avoids a proprietary agent on the database.
Metrics and logs together Metrics show that something changed; logs explain what happened. Debugging needs both.
Compute and database signals Host CPU and memory catch resource pressure; connection counts and replication delay catch database-level trouble.
Query-level statistics Per-query counters surface the slow or high-volume statements behind a latency spike.
Configurable authentication An API key, bearer token, or basic auth is needed to reach a secured collector endpoint.
Deep diagnostic capture Query plans, wait events, and schema-change history turn a symptom into a root cause.
Governed retention Keeping telemetry as governed tables lets you query history under the same access controls as your data.

How Databricks approaches this

Lakebase is Databricks' fully managed, serverless Postgres, and it is generally available. Observability is built in rather than bolted on. Each project runs its own managed OpenTelemetry Collector, and you turn on export from the Lakebase app under Settings, Integrations, OpenTelemetry integration. There you set the connection protocol, the endpoint URL, the authentication method (API key, bearer token, or basic auth), optional path overrides, and resource attributes that tag the telemetry. From that point the collector streams the metrics and logs described above to your platform. See the Lakebase OpenTelemetry documentation for the exported metric list and setup steps.
For diagnostics that go past dashboards, Advanced Postgres Telemetry is an opt-in option that captures deeper Postgres signals: active sessions and wait events, query plans, per-query execution statistics, schema and DDL history, database statistics, and raw log messages. It writes them as Delta tables in your own Unity Catalog, so the history lives beside your analytical data and inherits the same governance and retention you already apply. Every project also ships with built-in monitoring views: metric graphs covering up to seven days, the active queries from pg_stat_activity, the most frequent and slowest recent statements from pg_stat_statements, and a system operations log. See the Lakebase monitoring documentation and the general availability announcement.

Use cases

Unifying database and application monitoring. Teams that already watch their services in one observability tool point Lakebase at the same OTLP endpoint, so database health sits on the same dashboards and alerts as everything else.
Diagnosing a latency spike. When an app slows down, connection-count and replication-delay metrics show whether the database is the cause, and the query statistics identify the specific statements to fix.
Keeping an audit-friendly history. Writing Advanced Postgres Telemetry to Unity Catalog gives platform teams a governed, queryable record of query plans and schema changes over time.

FAQs

Which observability platforms can Lakebase send data to?

Any backend that accepts the OpenTelemetry Protocol (OTLP). The managed collector forwards metrics and logs to the endpoint you configure, so you can keep the monitoring tool you already run.

What metrics does Lakebase export?

Host metrics for the compute, such as CPU and memory, and database metrics such as connection counts, database size, replication delay, and row activity. Database metrics use a lakebase_ prefix so they are easy to isolate.

Does scale-to-zero affect telemetry?

Yes. Metrics and logs come from the active compute that serves an endpoint. If scale-to-zero suspends that compute, it stops emitting until a new request wakes it.

Can I keep detailed Postgres diagnostics inside Databricks?

Yes. Advanced Postgres Telemetry writes query plans, session and wait-event history, execution statistics, and schema-change history as Delta tables in your own Unity Catalog, so you can query and retain them under your existing governance.

Is pg_stat_statements enough for query analysis?

It helps. pg_stat_statements reports aggregated counters per normalized query, which is good for finding slow or frequent statements, but it is a statistics view rather than a per-execution audit log.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.