Building Your Open Security Lakehouse: Ingest, Normalize, and Model Security Data at Scale

Summary

Security is fundamentally a data engineering problem—applying medallion architecture to bronze, silver, and gold tables gives SOC teams a reliable, queryable foundation before detection work begins.
Lakewatch's reference architecture normalizes security, IT, and OT telemetry to OCSF, routes hundreds of log sources through a standardized routing table, and manages governance through Unity Catalog.
This video covers detection-as-code with collaborative peer review, asset and identity graph construction for threat correlation, agentic threat response with Agent Bricks, and when to augment versus replace existing SIEM tooling.

Building Your Open Security Lakehouse: Ingest, Normalize, and Model Security Data at Scale

Many organizations are experimenting with “security data lakes,” but struggle to move from DIY projects to an operational architecture the SOC can live in every day. this video walks through Lakewatch’s reference architecture for an open security lakehouse — how to onboard and unify diverse security, IT, and business telemetry on open storage, normalize it to OCSF, and organize it into bronze, silver, and gold tables optimized for SecOps. We’ll cover ingestion patterns for common data sources, Medallion modeling for security analytics, governance with Unity Catalog, index and search strategies, and how to safely bring in multimodal data like emails and documents. You’ll leave with an opinionated architecture to evaluate or design your own security lakehouse, plus guidance on when to augment versus replace existing SIEM tooling.
Talk By: Kristin Dahl, Cybersecurity Product Specialist, Databricks ;

Chapters

FAQs

What is an open security lakehouse?

An open security lakehouse consolidates security, IT, and business telemetry on open storage, normalizes it to a common schema like OCSF, and organizes data into medallion-layer tables optimized for SecOps workflows. Unlike proprietary SIEMs, this architecture gives teams full control over their data pipelines and detection logic.

What is OCSF and why does it matter for security data?

OCSF (Open Cybersecurity Schema Framework) is a normalization standard that unifies the format of logs and events from hundreds of different security data sources. Normalizing to OCSF at the silver layer means detection engineers and analysts work against a consistent schema rather than source-specific formats, accelerating detection engineering.

How does Unity Catalog support security data governance in Lakewatch?

Unity Catalog provides centralized, fine-grained governance across all security telemetry stored in the lakehouse, including table-, row-, and column-level access controls. This ensures sensitive security data is only accessible to authorized teams while maintaining complete audit lineage for compliance.

When should a security team augment versus replace their existing SIEM?

This video explains that pipeline reliability and data quality failures—not model quality—are the primary reason detection and response work stalls. A security lakehouse can complement an existing SIEM by improving data foundations and enabling advanced analytics, or replace it when cost and data limitations become prohibitive.

Full transcript

[00:08] Hi folks. Um How's everybody doing? Yeah. Um I just got back from maternity leave. It's my fourth week back and uh
[00:20] Thank you. Hitting the ground running. So uh Is Taylor here? My uh my colleague Taylor, when I got back from leave he said, "Kristen, I I volunteer you to give a talk at Summit. I hope you
[00:32] don't mind." It's like, of course. Uh I don't mind at all. I'm actually getting the the best sleep I've had in a few months. So it's uh it's been good. Um but uh
[00:45] I'm here to talk about security. So um who here, just show of hands, who here's involved in security or has a background in it, that's your job function? Okay. What about data engineering? A lot of Yeah, okay. Okay. Yeah, it's
[00:58] kind of like converging now. Um So I work in the I'm one of the um product specialists at Databricks. My background's cybersecurity. I've been with the company for just over 5 years.
[01:12] And I work with a lot of our customers, ranging from Fortune 500s to um small startups. And my job is to work with their SOC teams, their engineering teams to understand what hurts. Like, where's the
[01:24] pain? Where can Databricks help? And um My background is a a little bit different from a traditional um security analyst. So I actually come out of a formal methods background and
[01:36] systems background. So when I approach problems like this, it's through the lens of reliability in systems. Um Previously, I I I was research staff member at MIT Lincoln Laboratory. I did
[01:49] a lot of work uh R&D on with the DoD, DHS. And then I came to Databricks. One of the big problems, and actually what what drew me to Databricks, was this this notion that security is a data
[02:01] problem. You know, I think we all know that. Um we're dealing with, you know, hundreds of different data sources, ingesting it at various volumes. Before we can actually reason about the data, perform like build detections on the
[02:14] data, we have to we have to get the data in a in some kind of shape that we can understand. And uh I kept running into the same problems where folks would get tripped up on the pipelines, they'd run into
[02:26] issues where their data was a mess, and it it was very hard to even get to the detection and response phase. Um So, when I came to Databricks, I had the idea, you know, we should just take
[02:38] these principles from data engineering and point them to security. You know, there's not one security tool that's going to fix this all, but we need to uh approach this from a a more principled way. Um and so, what I'm going to talk
[02:51] through today is kind of the conversations that I have with a lot of my customers, and I'm going to point out uh where I see a lot of problems happening, and, you know, what what um what I would typically recommend as
[03:03] solutions, and uh I'll I'll tie this all back. I don't know if I'm sure you've all seen or heard of Lake Watch by now. Um Michael Anderson did an awesome demo this morning of our our new Lake Watch product.
[03:15] And at the end, I'll kind of tie this into um what we're doing with Lake Watch. Okay? So, why do security data lakes fail? You know, I've seen this done time and again. You
[03:27] know, I'm I'm I'm guilty of building these dumpster fires where you basically dump all your data into raw storage, and it sits there, and you build these parsers, you build like a a parser for
[03:39] every data source, and then you have to have like individual parsers for different versions of the data sources. Then you have to hold your breath when a pipeline breaks, or you you deploy a change. It's very brittle system.
[03:51] Um the other issue is run in we run into is tool lock-in. So, you've got detections that are tied to one tool, so you're kind of like wedged into one ecosystem. That's not a great place to be. You know, you're you're kind of held
[04:03] hostage. Um and then the governance piece, this is kind of a lot of the times this isn't really thought through at the beginning and so you wind up bolting it on at the end. You end up with a very fragile brittle
[04:16] system and that's not the place that we want to be. We want to be on the right-hand side. You know, where we've got a queryable normalized source of truth that we trust. Um resilient pipelines that survive things
[04:29] like schema drift or uh malformed fields or whatever other problems that happen in the data ingestion um in that process. Portable detections, so being able to um
[04:42] kind of swap out the tools underneath without having a lot of operational friction. And then we governance and the the um retention, the provenance, all of that built in at the foundation level. That's
[04:55] what we're looking for. Okay. So, but the gap between, you know, the left-hand left and the right is kind of um the difference between a dumpster fire and a
[05:08] real operational architecture. And so, um let's walk through kind of this whole process. So, we'll start with with ingest.
[05:20] Um I think we've all been here. So, this is the unglamorous part that everyone underestimates. You know, people want to hit the ground running. They want to start like building detections across all of the data. But this is the hard part. Um
[05:32] and so, let's talk about how we actually get um our data into open storage without making a mess, okay? Um first principles. Like I said, I like to root things in first principles.
[05:46] Um if you've worked with Databricks at all, you probably recognize this medallion architecture. It's a multi-hop architecture. We're landing our bronze data on the left-hand side, and this can be any sort any of your sources,
[05:59] structured, unstructured. Land it in what we call bronze. It's our first hop. We append only, right? So, the the raw data sits here. And then from there, we can clean it up, we filter it, we do some transformations
[06:12] on the data. We turn those into what we call silver, our second hop. This is usually where we'd build our detections from are these silver tables, okay? >>
[06:25] case dependent. So, these are going to be shaped around how you're using the data. So, think about like threat hunting, investigations, dimension tables, they'd all be classified as gold tables, okay? So, at a very high level, this is kind of the flow that we want.
[06:40] First big problem I run into is the ingestion itself, okay? Very important that we think about um the ingestion method that we're using for the source. So, I get a lot of
[06:53] customers who are like, "Oh, we need sub-second latency. It doesn't matter. We we absolutely have to have it across all of our data sources." You know, or we only use batch, we only use autoloader. I I This is kind of the This is the discussion I have with almost all of my
[07:06] customers is um Think about this as you've got one destination, so I got Databricks in it's a open cloud storage. That's your destination. You've got multiple on-ramps to get there.
[07:19] Okay. So, it's very important to think about um which on-ramp fits your data source. So, for example, if you've got like a SAS or security tools, you're probably going to use something like I don't have it on there, but like ZeroBus or like Flow Connect.
[07:32] If you've got files in cloud storage in S3 buckets or something, that's a great candidate for autoloader. Uh structured streaming, obviously if you guys streaming data sources, sub-second
[07:44] latency. Um, that's when you'd use structured streaming. So, the the whole point of this is you've got, um, the the source will kind of dictate to you which on-ramp to take.
[08:00] The second is when you land your data, don't touch it. Resist the urge to do any kind of clean-up or touching, like transformations on the data. This is very important for a number of reasons. Number one, um, if, uh,
[08:14] if say down the line, you've got a normalization rule or something, it's a transform, and the, uh, the you you six months from now you realize, oh, you know what? That rule is wrong. We made a slight mistake. We need to rerun our pipeline.
[08:27] If you had transformed that data, if you had modified it, uh, you can't actually go back to your ground truth and reprocess that pipeline. Right? That becomes a data loss incident. Very important, especially if we're
[08:39] dealing with any kind of auditing, compliance, financial, um, information. Um, so what I recommend is just save this as, uh, raw data. But also add this metadata envelope.
[08:52] Okay. So, what is that? So, in addition to saving the raw payload, I like to tack on a few fields. Number one is the time that we ingested the data. So, when did we actually ingest this log?
[09:04] The second is the, um, the source timestamp or event time. So, in the log, like when did the actual event occur? Those are two different timestamps. The source system, where did it come from? It could be AWS, could be Octa,
[09:17] could be, um, Azure. The source type, um, would be like CloudTrail. If it's API data, it'd be API, things like that. And then the batch ID, this is a unique
[09:29] ID that you just generate and stamp on to each of these, uh, entries. Just for traceability and auditability, or if you have to, like reprocess a pipeline, it makes tracking a little bit easier.
[09:41] Okay. So, that's the bronze. So, here's an example of um leveraging autoloader to ingest some cloud uh JSON files into uh just a
[09:53] Delta table. Okay, it's super simple. Um you'll notice I've got a an option set for uh add new columns. So, what that means is we're going to evolve our schema. So, if down the line our vendor decides
[10:06] to add a new field or modify the schema, our pipeline's not just going to fall over. It's actually going to modify it's going to adapt to those new additions to the data. Um you can see there in the middle, that's just where I'm adding on those
[10:18] metadata columns, and then we're writing it out. Uh super simple. This is kind of like the boilerplate code that I give to a lot of customers as an example.
[10:31] Um the last piece that I want to talk about on the ingest is governance. This is overlooked quite a bit. Um and it's super important that you think about it up front before you start building this out um because you don't want to have to go back in time and figure out like what
[10:44] do I have to change, what permissions. Um and I like to break it into four buckets. Okay, so I think about in terms of protection. So, how do we protect the data? How do we protect like things like PII and whatever from
[10:57] um being uh being accessible by I don't want to say the wrong people, but people that don't have permission to look at it, right? So, the protection piece, preservation.
[11:10] >>
[11:22] to lineage as your data flows through your pipeline. Okay? So, for example, if I've got a detection and it fires, I want to have confidence to be able to go back and understand how is that data transformed? How did that detection
[11:35] fire? Okay. Um that can tell me we can go back and understand um the the logic that happened to to um to fire that detection.
[11:48] Um and then the last piece is the uh discoverability. Um so this if you've ever had a situation where you've got like tribal knowledge where you know somebody who knows everything just like up and leaves,
[12:00] uh and you you lose a bunch of knowledge or if you're relying on Confluence
[12:12] you do have the ability to provide comments and metadata and more context around your tables. And this is accessible by humans or also agents. And I know if you if you listened to some of the keynotes today, we talked about or
[12:25] Ali talked about the importance of context, establishing context within your environment. So that's we say catalog or discoverability, that's what that refers to.
[12:37] Okay. So the second piece is normalization. And this >>
[12:50] I know that's I've I've really been in heated conversations. Um when we talk about normalization, we talk about the I'm I mean things like modeling the data. So OCSF, ASIM, common
[13:02] information model. There's a whole bunch of different ways you can model the data. Um and I think they're all wrong. Like fundamentally they're all wrong because when you model the data, you're losing a you're
[13:15] losing a bit of the truth, right? You're taking an event and you're trying to map it to some subjective model of what reality is, okay? Um I've been on conversations with customers where
[13:28] um I remember one in particular this person was very frustrated because their manager wanted them to take um I think it was CrowdStrike data. They wanted them to map every single field of CrowdStrike
[13:40] to um to an OCSF uh class or identifier. You've ever used CrowdStrike, like raise your hands. It's not nice, right? It's It's not fun to work with. It's It's
[13:53] very complicated. And so my advice is to treat this as a baseline, okay? Um no standard is perfect, okay? The right choice is going to depend on your
[14:06] existing tooling. Uh it's going to depend on the diversity of your data sources, your your own environment, your preferences of your users. And so treat them as a baseline, don't treat them as gospel. Um and then be ready to bend or break
[14:19] the rules if they get in the way of solving the real problem. Okay. Mapping meaning. So this is one of the mindset shifts that I try to instill in folks when we're
[14:31] doing one of these modeling exercises is we're not actually like doing like a column renaming or trying to um map like columns to specs. That's not what we're talking about. We're talking about the semantics of these logs.
[14:45] What do I mean? So if we look on the slide, we've got CrowdStrike logs, you've got Okta authentication, we've got um Zeke and firewall logs, right? These all have some concept of an
[14:57] authentication event, of a session, of a user. Now, but if we were to write detections against all of these, it would become messy very quickly um because we'd have to write individual detections for each log source, and then
[15:10] we'd have to update them all and maintain them all. And certainly there will be exceptions where, you know, we we would want to do that, but for this exercise, let's pretend we want to map them all to OCSF. Okay, so all of these have some concept
[15:22] of a user, right? A device, a source IP, and a status, so success or failure. That's enough to write a simple detection. Okay. And when we talk about
[15:34] making detections portable, I don't know if you've heard that phrase or not, this is what we mean. Okay. Now, you do run into problems. Like I said, every, you know, modeling's aren't
[15:46] perfect These models aren't perfect, and so >>
[15:58] were to try to This is a toy example, but let's say we wanted to map this to our authentication class, we would lose data. We would lose some of the context because there's only room for one person
[16:10] in OCSF. There's only room for one user. Okay. So, now we we'd either have to drop Bob or Alice, and then we wouldn't know is this a admin um initiated MFA reset or a user-initiated
[16:23] reset. That's very important. So, the OCSF answer would be like, well, no, you know what? This is actually an account change, event 3002. That captures two users. Okay.
[16:35] So, if we were to put put this in production, it does require a little bit of legwork because we'd have to understand what are the different What's the landscape of the events that we want to capture,
[16:47] and then how are we mapping them to OCSF or ASIM or whatever our model of choice is. It's some work up front, but it does pay off. I think it's absolutely um very useful to go through that
[16:59] exercise. What I recommend is build a routing table. It does It's not that hard. You basically It's a dictionary where you're keying off of the event type and mapping it to whatever your event class is.
[17:12] Um and then if you've got unmapped types, what you can do is actually put them in a silver table. Just put them in a table and then alert on them. So any unmapped
[17:24] and can be updated in the router table. Okay. Now the gold common kind of misconception I had is that these are like silver 2.0 or you're just like it's
[17:37] cleaner silver. That's not necessarily the case. >>
[17:49] need to act on? All right. And I've I've got some common examples that I see up there. So they they alerting tables anytime you have a detection that pops goes into the alerts table.
[18:01] Um cases table if you're doing an investigation that gets uh kind of saved off in our cases table. Dimensions table, identity table uh which I'll talk about in a little bit. >>
[18:14] got like OSINT or threat intelligence that you're periodically updating. Uh I like to have it in its own table. And these are tables that like humans or agents can use as part of their investigations.
[18:29] Now this is probably um we we've all kind of experienced this where you've got one human identity but then multiple um uh uh ident like identifiers within your environment for that one person.
[18:42] And it can be a real pain if you ever if you're investigating an alert and you only have like an IP address. You start had you have to pivot across all of these different data sources to put the context together, right? This IP
[18:55] belongs to this host. Who is logged into the host? Where are they logged in from? All of this information you have to piece together. I highly recommend if you you if you haven't done this, put together this um
[19:07] identity table. It's a asset and identity graph. Basically mapping all of this information together. Okay. Um so, this is a dimension table and we're basically um resolving resolving
[19:21] all of these aliases down to one canonical entity. How do we do that? Again, like the modeling, this is a little bit of work up front, but it pays off like massively uh downstream.
[19:37] So, step one. And I mean, this is easy to talk through, but it um I like to recommend start with your strong keys first. What's that? These are things that are not necessarily time bound. So, corporate email address, Okta
[19:51] ID, your active directory. These are all things that are pretty stable that aren't going to change very frequently for your user. If you do just that, you're 80% of the way there. Okay, if you just do that. The second
[20:04] step are the weaker signals. These are going to be time windowed. Okay, these are ephemeral things like um the person like the an IP address that belongs to a person for the duration of a DHCP like lease or
[20:16] something. Um or specific um host names that might change. And then you set that to update every night or whatever frequency makes sense. Where you fold in the new new entities
[20:30] and then you phase out the the old ones after like 90 days or something if there's no updates. Okay. Ingest. Normalize your data. Now, we can
[20:42] actually get to DNR. This is the stuff that like security practitioners actually want to do, but they have to get all the other muck out of the way first. Um so, on Databricks,
[20:55] detections are SQL. Um we don't have like a special query language or anything that we make you learn. Just SQL. Um all the detections are performed where your data lives within your
[21:07] lakehouse. Um you're using the same languages for batch or streaming queries, and it's all version It can all be version controlled. So, you can set up like um detection as code pipelines. You can
[21:19] easily peer review these detections. Um you can set up um really uh interesting like CICD pipelines for your detections and stuff. Um this is just an example I put together. It's very
[21:31] simple. It's a brute force authentication attempt attempt in SQL. If you don't know SQL, um if you're like me and you can like anything beyond like a
[21:43] select star is intimidating, and I I count myself among that group. I'm not a a SQL expert. Um Genie can help tremendously in this space. Um and this is coming from me. I'm a
[21:56] I'm like an AI skeptic or I I was very early, and I'm like, "Oh, this is never going to work." It actually is like it's pretty It's pretty good. Um you basically take your plain English and you pop it into Genie, and it will
[22:08] give you um kind of like a what I call a first-pass query, okay? Cuz I like to go in. I like to twiddle the parameters and like change things around, but it'll get you like 90% of the way there. How does it do that?
[22:21] Um you remember we talked about discoverability and context, and how important it is to set that data metadata around your tables. Genie will actually access that information. It'll look like at the data types that you're querying. It'll look
[22:33] at the type types of tables you have. It'll look at the context around the tables, and it'll use that as it starts to build these queries. Um this is great for engineers who don't know SQL
[22:45] or uh maybe you know SQL, but you're not sure where to get started. This is a great way to get started. Um and we're not trying to like remove the analyst so much as speed up workflows and remove that barrier to to building detections.
[23:04] Okay. If you take If you're going to take a picture of a slide, this is the one. I'm serious. This is the one that caught I'm like I could do an entire session. I probably should have. Do an entire I you know, I would have, but Taylor signed me up for this talk, so I have to
[23:16] do this talk, but I If if I signed myself up for a talk, it would have been for this. Um Bec- This is so I spend so much time with customers trying to unders- trying to build out
[23:29] just like this detection and response um uh loop. And so if if I were to do it today without Lake Watch, so caveat this is without Lake Watch. The way that I would do it is anytime I
[23:42] set up my SQL um detections, anytime one pops, alert goes into your your alerting table. And that's become That becomes your system of record for your for your alerts. Okay.
[23:55] From there, um it the second phase is the operational layer, so that's um looking at deduplication, suppression, performing any kind of like triage on
[24:07] the alerts, um enrichment. And this is actually a great use case for Agent Bricks, and I I'm starting to see this a lot more is what can we use Agent Bricks for to bring in some additional threat intelligence, some additional OSINT, um
[24:20] are Do we have prior cases that we want to correlate with this one? Great use case for Agent Bricks. And then you can make a decision. Do we ticket this? Do we send it off into our SOAR? Um is this a false positive?
[24:34] Um And then um you can you you can use the the tools that you're already using, whether it's ServiceNow or uh Jira or SOAR, whatever it is. You take that ticket number or whatever
[24:47] that decision is, and you'd pop it back into your cases table. Remember, we've got that gold cases table that we're using to manage our investigations. So all that would go in there. Um and then the disposition. So, what
[24:59] did the analyst decide? You know, is this something where we needed two in the rule? Was this a false positive? Um, was it something where we need to actually follow up and create some additional rules? Um, that would go in our cases table. Or wait, no, our feedback table.
[25:13] And then periodically we could go through and tune our rules and that's kind of the feedback loop. Now, mapping
[25:26] um, confusing to figure out like, well, what it what should I be using for what? Um, Lakehouse jobs, this is like the operational piece. Um, Agent bricks, you can and we're doing this right now
[25:39] within Lakehouse if you saw some of the demos, using Agent bricks to triage and enrich. And then Delta and Unity Catalog would keep uh, be the system of record. And then you'd still use your existing
[25:51] tools um, for any kind of like an additional follow-up or automation that you you'd want to do. Okay. So, let's see.
[26:05] Wrap up. So, we started the talk with um, why are why is my security like a a dumpster fire and we went through the reasons why. Um,
[26:17] and what and then the question was, well, what do we actually need for an operational security lake? And I argue that there's four points. You know, >>
[26:30] to be able to trust our data. We get that through with the modeling and the the detections that we're building in our silver tables. Um, ingestion that survives schema drift.
[26:42] So, that's resilient pipelines. We don't want to have to be like fight like cleaning up pipelines and fixing things constantly. We want pipelines that are actually operationally resilient. And we get that through saving our data in
[26:54] bronze, making it append-only, not changing it. Um portable detections that outlive any one tool, we get that from using SQL instead of like a proprietary uh query language, we're
[27:07] just using standard SQL. Um and then the governance piece, we get that with Unity Catalog, at least on Databricks. And that's built in at the foundation. And so we get that for free. Well,
[27:19] um so the last slide um is the economics because I some people care about money. I it's it's not my job, but you know, I I do have to talk about it. Um
[27:31] but I have seen so across the customers that I have worked with, I have seen a significant cost savings in moving from a traditional SIM architecture to something akin to what we just walked through.
[27:43] Um and also huge performance increase. And there's a couple of reasons for that. Um on the the the ingest piece, now if we'd look at a traditional SIM with their the pricing model, it's usually
[27:55] based on how much data you're ingesting. Okay? So what that means is your retention becomes a budget line item, and you have to make a decision about which logs can I keep and for how for
[28:07] how long. Okay? So that puts a a limit on your observability. Like fundamentally, you can keep like X amount of logs, and um hopefully, you know, in the the
[28:19] the the logs you're throwing away, you're not missing something, right? Um and if you want to try to do any long-term historical analysis or trend analysis, um good luck. You're not going to be able to do that. Um
[28:31] the performance piece, I'll have a uh comment on there, but the the performance piece, a lot of these older uh SIMs were not built with a scale in mind for the the the volumes of data
[28:43] that we're seeing today. They just they're not able to meet that that volume of data. Okay. So, on the lakehouse um the the the benefit here is it's you're separating your compute and your
[28:55] storage. All of your data is stored on your a cloud storage, which is very economical. It's much cheaper. And um you're only paying for the compute that you're using. So, when you're running queries or running an ingest job, that's when
[29:08] you would actually incur payments. Um the benefit here too is that we're retention stops being a rationing decision. You can keep as much data as as you want. I mean, there are practical limitations, but in theory, you you
[29:21] could. Now, there's if you if you go on our website, and if you I can provide some some more more numbers on this too if you you you email me or come up to me. Um
[29:33] but if we were to take like let's say the cost of like a regular SIM is 100. Okay, I've seen this reduced by factor of like down 75% in one instance. Normally, it's like 50 to 60% I see a
[29:45] reduction in in overall spend. And I've seen queries uh ingest latencies go from like 15 minutes down to a minute. Um queries that finish in a matter of seconds, and then overall uh
[29:57] SOC productivity improvement because you're um you're spending less time waiting for the queries, you're spending less time like running around chasing context and trying to to to pull all these different logs together because
[30:09] everything is is right there. >>
[30:22] of steps. And that's why we have That was the impetus for Lake Watch. So, um like I've said, I've been with Databricks for um just over 5 years. I was actually one of
[30:35] the the first dedicated cybersecurity product specialist we have. So, that the architecture that I walked through, we were implementing that time and again across a lot of different customers.
[30:48] And it it really started to catch on and we we started to notice a trend. More and more customers were asking us about it, like how can we actually do um cyber security on Databricks? And they'd see some of the talks like Rivian did a talk a couple years ago on their
[31:01] trail system, which is really good. And so it really start it started to catch on. And so internally we were like, we need to productize this. We need We need to make it easier for folks to do this.
[31:13] And so that's what Lake Watch is. And so if you've If you've seen the demo that the demo's down in the Expo Center, too, if you want to go take a look. Um everything I talked about, building the pipelines, the the the normalization,
[31:25] the tables, the cases, the the queries, it's all part of Lake Watch. We're We're trying to make it more of a curated, easier experience for you. So you're not going in there and having to like
[31:37] manually set this up yourselves. Okay. Um it's still in private preview. Um so we we do have some design customers that were partners that were working with and they've been awesome at testing it and providing feedback. Um
[31:51] but I'm really excited about where this is going to go, um especially with some of the agentic capabilities that we're starting to build into it. I think we're going to be able to do some very interesting things. Um
[32:04] so that being said, um my takeaways, you know, security's a data problem. We should solve it like one. And we have a few minutes left, so love to take some questions.
[32:16] Reach out to me on email, too, if you if you want to connect and have another conversation. Um always happy to chat and collaborate on things.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.