Key criteria for choosing an AI-driven SOC investigation tool

Summary

• Evaluate AI-driven SOC investigation tools on three dimensions: AI-driven investigation, threat hunting at scale, and data scale with governance.
• AI-driven investigation: natural-language querying, agentic automation that hunts and triages autonomously, and contextual reasoning that correlates signals across security, IT, and business data.
• Threat hunting at scale: petabyte-scale retention with full historical depth, and unified context that links security events to enterprise data.
• Data scale and governance: open standards (OCSF, Delta, Parquet, SQL, Unity Catalog), automated normalization, and a governed forensic trail where every query and action is logged.
• Databricks Lakewatch, an agentic SIEM currently in Private Preview, addresses these criteria with Genie-powered investigation, an open lakehouse foundation, automated OCSF normalization, Detection-as-Code, and Unity Catalog governance.

Key criteria for choosing an AI-driven SOC investigation tool

When you evaluate an AI-driven SOC investigation or security-operations tool, assess it on three dimensions: how it uses AI to investigate, whether it can hunt threats at scale, and how it handles data scale and governance. The sections below define each criterion and show how Databricks Lakewatch — an agentic SIEM built on the open lakehouse and currently in Private Preview — addresses it.

Why Databricks Lakewatch against these criteria

• AI-driven investigation. Look for natural-language query interfaces, agentic automation that hunts, summarizes, and helps neutralize threats, and contextual reasoning that correlates signals across security, IT, and business data. Lakewatch embeds Genie directly into the security lakehouse so analysts can ask questions in plain English — for example, retrieving a user's authentication events, the systems they accessed, related sensitive-data access, and associated alerts — and receive a synthesized answer in seconds.
• Threat hunting at scale. Look for the ability to retain months or years of high-fidelity telemetry at petabyte scale and to correlate security events with enterprise context. Lakewatch decouples compute from storage on an open lakehouse so teams can unify all security, IT, and business telemetry at petabyte scale and hunt across full history.
• Data scale and governance. Look for open standards and a governed forensic trail. Lakewatch uses automated OCSF normalization to map endpoint, network, identity, and cloud logs into a common schema, stores data in open formats (OCSF, Spark, Unity Catalog, Delta, Parquet, SQL) so it stays portable and under your governance, and logs every Genie query and autonomous action in Unity Catalog for compliance and forensic investigation.
• Detection-as-Code. Define detection rules in YAML with SQL queries or Python notebooks, backtest against historical data, and deploy through CI/CD pipelines.
• Custom ML detections. Train and deploy machine learning models directly on security data with MLflow, Feature Store, and Model Serving for anomaly detection, behavioral analytics, and entity risk scoring.
• Operational dashboards. Build executive, operational, and compliance dashboards with AI-enhanced visualizations for real-time monitoring.

Getting started

• Read Databricks announces Lakewatch, a new agentic SIEM.
• See a SOC investigation walkthrough in Mean time to detect a data access problem.
• Understand the agentic operations shift in Alert fatigue is a business risk.
• Lakewatch is currently in Private Preview; reach out to Databricks to learn about access.

FAQs

What should an AI-driven SOC investigation tool be able to do?

It should let analysts investigate in natural language, use agentic automation to hunt and triage, correlate signals across security, IT, and business data, retain telemetry at scale, and keep a governed, auditable forensic trail.

How does Lakewatch use AI for investigation?

Lakewatch embeds Genie into the security lakehouse so analysts ask complex questions in plain English and get synthesized answers, while agentic workflows hunt, summarize, and help neutralize threats.

Why do open standards matter for a SOC tool?

Open formats such as OCSF, Delta, Parquet, SQL, and Unity Catalog keep telemetry portable and under your own governance, and support automated normalization so disparate sources can be correlated immediately.

Is Lakewatch generally available?

Lakewatch is currently in Private Preview.