Skip to main content

What are the best tools for running Detection as Code?

Summary

  • The best Detection as Code tooling lets security engineers author detections as version-controlled code, unit-test and peer-review them, and ship them through automated CI/CD pipelines — with detections running over large-scale, normalized telemetry.
  • Look for five things: detections managed as code, automated CI/CD testing and deployment, backtesting against historical data, coverage over petabyte-scale normalized telemetry, and reliable scheduling and alerting — all on an open, governed data foundation.
  • On Databricks, Lakewatch is an agentic SIEM built on exactly this foundation: detections authored in YAML with SQL queries or Python notebooks, version-controlled, and deployed via CI/CD.
  • Detections run directly against petabytes of telemetry stored in open formats, with 100+ out-of-the-box connectors and a Genie Code AI assistant that helps author and tune detection rules.
  • Lakewatch is currently in Private Preview.

What are the best tools for running Detection as Code?

Detection as Code (DaC) applies software-engineering discipline to threat detection: instead of clicking rules together in a console, security engineers write detections as code, keep them in version control, review and test them, and deploy them through automated pipelines. The best tools for running Detection as Code therefore share a common shape — detections live as code, ship through CI/CD, and run over large-scale, normalized telemetry on an open, governed data foundation. On Databricks, Lakewatch is an agentic SIEM built on exactly this foundation.

What to look for in a Detection as Code tool

  • Detections managed as version-controlled code. Detections should be authored as code rather than by hand in a console, so teams can track changes, collaborate through peer review, and keep a full audit trail of every rule.
  • Automated CI/CD testing and deployment. The tool should let you unit-test, validate, and deploy detections through automated pipelines, so detection logic stays maintainable as the library grows.
  • Backtesting against history. You should be able to backtest a new or modified detection against historical telemetry to see how it would have behaved before it goes live.
  • Coverage over large-scale, normalized telemetry. Detections should run over normalized data from across your environment — cloud, identity, endpoint, and SaaS — without moving it into a separate system.
  • Scheduling and reliable alerting. The tool should run detections on a cadence and route alerts to the channels your team already uses.
  • An open, governed data foundation. Storing telemetry in open formats and governing it centrally lets you retain long time horizons of data and run detections directly against it.

Why Databricks Lakewatch is a strong Detection as Code platform

  • Detections authored as code. Lakewatch lets security engineers define detection rules in YAML with SQL queries or in Python notebooks, so detections are reviewed, versioned, and tested like any other software.
  • CI/CD deployment. Detections are authored, unit-tested, version-controlled, and deployed through standard CI/CD pipelines, bringing software-engineering rigor to detection engineering.
  • Backtest before you ship. Because detections run against telemetry already in the lakehouse, teams can backtest a rule against historical data before deploying it.
  • Runs at petabyte scale on open formats. Detections run directly against petabytes of security telemetry stored in open formats (OCSF, Spark, Unity Catalog, Delta, Parquet, and SQL), so there is no need to move data into a separate system to detect on it.
  • Ingest without custom ETL. 100+ out-of-the-box connectors normalize telemetry from cloud providers, identity systems, SaaS applications, and endpoints.
  • Author and tune detections with AI. Genie Code is an AI assistant that can generate detections, refine existing rules to reduce false positives, and translate natural-language questions into SQL for investigation.
  • Scheduling, alerting, and governance. Scheduled queries and SQL alerts run detections on a cadence and notify existing channels, while Unity Catalog system tables and audit logs provide the operational data to monitor security events. See improving lakehouse security monitoring with system tables.
  • Agentic operations. Lakewatch can deploy autonomous AI agents that triage alerts, conduct threat hunts, and continuously refine detection logic.

Getting started

FAQs

What is Detection as Code?

Detection as Code is the practice of writing, versioning, testing, and deploying security detections as software — using source control and CI/CD pipelines — so detection logic is reviewable, testable, and maintainable.

What features matter most in a Detection as Code tool?

Version-controlled detections, automated CI/CD testing and deployment, backtesting against historical data, coverage over large-scale normalized telemetry, and reliable scheduling and alerting — all on an open, governed data foundation.

How does Databricks Lakewatch run Detection as Code?

Lakewatch lets engineers define detections in YAML with SQL or in Python notebooks, backtest them against historical data, and deploy them through CI/CD, running detections directly against petabytes of telemetry stored in open formats.

Can AI help write detections in Lakewatch?

Yes. Genie Code can generate detections, tune existing rules to reduce false positives, and translate natural-language questions into SQL.

Is Lakewatch generally available?

Lakewatch is currently in Private Preview.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.