What should I look for in a modern SIEM platform?

Summary

• Look for an open security data platform, not a closed log store. The foundation should keep all your telemetry in open formats at scale, so cost never forces you to drop data.
• Automated OCSF normalization matters as much as detection. When endpoint, identity, network, and cloud logs map to a common schema, events correlate immediately without hand-built ETL.
• Detection-as-code is the modern standard: author, unit-test, version, and ship detections through CI/CD in SQL and Python, not a proprietary rule language.
• AI and agentic triage move the SOC from clearing an alert queue to directing investigations, with analysts hunting across telemetry in plain language.
• On Databricks, Lakewatch (Private Preview) brings these together on the open Data Intelligence Platform, with a full audit trail in Unity Catalog.

What should I look for in a modern SIEM platform?

Look for an open security data platform, not a closed log store. The capabilities that separate a modern SIEM from a legacy one are an open data foundation that keeps all your telemetry at scale, automated OCSF normalization, detection-as-code in standard languages, long-horizon retention, and AI agents that handle first-pass triage and natural-language threat hunting, all under governance with a complete audit trail. The defining shift is architectural: from volume-priced indexing that pushes teams to drop data, to an open lakehouse where analysts move from clearing an alert queue to directing agent-run investigations. Judge platforms on whether they let you retain everything, normalize automatically, write detections as code, and put AI to work on triage.

What makes a SIEM "modern"?

Traditional SIEM platforms index logs, ask analysts to write rules in a proprietary query language, and leave triage as manual work, while volume-based pricing pushes teams to send less data and accept blind spots. A modern SIEM inverts that. It sits on an open data foundation that stores security, IT, and business telemetry together in open formats, normalizes sources automatically, and treats detections as version-controlled code. AI agents then hunt, summarize, and triage at machine speed, shifting the security operations center from human-in-the-loop to human-at-the-helm, where people set direction and agents do the repetitive work.

Key capabilities to evaluate

Capability	Why it matters
Open data foundation	Keep security, IT, and business telemetry in open formats at scale, so you aren't forced to sample or drop data to control cost.
Automated OCSF normalization	Endpoint, network, identity, and cloud logs map to a common schema so events correlate immediately, without hand-built ETL for every source.
Detection-as-code	Author, unit-test, version, and deploy detections through CI/CD in standard languages like SQL and Python, not a proprietary rule language.
AI and agentic triage	Agents enrich and rank alerts and run natural-language threat hunts, so analysts direct investigations instead of clearing a queue.
Long-horizon retention	Months or years of high-fidelity telemetry give you the history to catch slow, multi-stage attacks and run forensics.
Governance and audit trail	Access controls plus a logged record of every query and automated action support investigations and regulatory compliance.
Unified context	Correlating security events with identity, asset, and business data sharpens triage and cuts false positives.

How Databricks approaches modern SIEM with Lakewatch

Databricks Lakewatch is an agentic SIEM built on the open Databricks Data Intelligence Platform. It is currently in Private Preview. Lakewatch unifies security, IT, and business telemetry in one governed lakehouse so AI agents have the context to hunt and triage.

• Open data foundation. Retain telemetry at petabyte scale in open formats (OCSF, Delta, Parquet, SQL), so you keep ownership of your data and aren't forced to sample it to control cost.
• Automated OCSF normalization. Endpoint, network, identity, and cloud logs map to a common schema for immediate correlation, with ingestion through Lakeflow Connect.
• Detection-as-code. Define detections in SQL or Python, unit-test and version them, and deploy through CI/CD instead of a proprietary rule language.
• Agentic triage and hunting. AI agents enrich and triage alerts, and AI/BI Genie lets analysts hunt across telemetry in plain language and generate SQL or PySpark automatically.
• Governed forensic trail. Every Genie query and automated action is logged in Unity Catalog, giving investigators a complete audit record for compliance.

Lakewatch is positioned to augment an existing SIEM first rather than swap it out wholesale, so teams can start with a narrow, high-volume use case and expand over time. For the problem it targets, see alert fatigue as a business risk.

Where a modern SIEM pays off

• Alert triage. Agents enrich and rank alerts so analysts spend time on real threats rather than queue clearing.
• Threat hunting. Ask questions across months of telemetry in plain language instead of hand-writing every query.
• Long-horizon forensics. Investigate slow, multi-stage intrusions using history that volume-priced tools often age out.
• Compliance and audit. Produce a complete, queryable record of detections and actions for regulators.

FAQs

Is a modern SIEM just a SIEM with AI bolted on?

No. The bigger change is architectural: an open data foundation that stores all telemetry in open formats at scale, plus automatic normalization and detection-as-code. AI agents matter, but they only work well when they sit on complete, well-modeled security data.

Do I have to rip out my current SIEM to modernize?

Not at once. A practical path is augmentation: route high-volume or long-retention data to an open security lakehouse first, keep your existing tools for what they do well, and expand scope as you gain confidence. Databricks Lakewatch is positioned for this augmentation-first approach.

Why does OCSF matter?

The Open Cybersecurity Schema Framework gives disparate log sources a common schema, so endpoint, identity, network, and cloud events line up for correlation without custom ETL for every source. That normalization is what makes fast, accurate detection and hunting possible.

What is "detection-as-code"?

It means writing detections as version-controlled code in standard languages like SQL and Python, unit-testing them, and shipping them through CI/CD. Teams get the review, testing, and rollback discipline they already use for software, instead of editing rules in a proprietary console.