From Vulnerability to Verified Fix: Governing Security Agents with Omnigent and Databricks

Summary

Proposed vs. Verified Fixes: A passing test written by an AI agent does not guarantee security; fixes must be independently tested against real-world attack vectors in isolated environments.
Multi-Layer Governance: Effective AI security relies on combining model reasoning, execution harnesses, and specialized cyber workflow skills structured into rigorous verification frameworks.
Human Authorization: Prevent automated agents from overstepping permissions by requiring human approvals to authorize scoped repairs and validate final patches before production deployment.

From Vulnerability to Verified Fix: Governing Security Agents with Omnigent and Databricks

Talk by Sapta Banerjee, Applied AI Engineer, OpenAI; Tom Petrick, Applied AI Engineer, OpenAI
A proposed fix from an AI agent is not a verified fix: governing cyber agent teams means independently reproducing and testing each patch, with a human approving scoped repairs. This video shows how Omnigent, a meta-harness from Databricks, coordinates OpenAI Agents SDK and Codex agents with an independent Databricks app verifier to investigate, reproduce, patch and verify a path traversal vulnerability.
OpenAI applied AI engineers describe three layers (model reasoning, the Codex harness and Codex Security workflows) and a five-stage governed workflow. In the demo, a security investigator agent produces findings, a Databricks verifier app reproduces the attack in a sandbox through an OAuth-protected endpoint, and a deliberately incomplete patch is caught when an encoded path still exposes another company's report. A Codex patcher then fixes it and the verifier confirms all nine requests return 403. They also cover MLflow traces for debugging multi-turn agents and evaluation flywheels that prevent regressions.

Chapters

FAQs

Why can't we trust an AI agent's own tests to validate security fixes?

Agents may write tests only for narrow use cases, missing alternative attack vectors. This video explains why independent verification against real-world attacks is mandatory for security patches.

What is the multi-layer governance architecture for AI security agents?

It combines model reasoning, execution harnesses like Codex, and specialized cyber skills in structured workflows. This video details the five-stage framework used at Databricks and OpenAI.

How should enterprises manage human approval of agent-generated security fixes?

Require human authorization for scoped repairs and final patch validation before production. This video demonstrates how to prevent agents from overstepping permissions and making unauthorized changes.

Why is observability crucial for autonomous security agent teams?

Deep execution tracing with tools like MLflow and evaluation flywheels identify failure modes and prevent regressions. This video covers continuous improvement strategies for security agent workflows.

Full transcript

`00:00` Introduction

Okay, hey everyone. So, our today's session is from vulnerability verified fix. So, I'll we'll give a quick introduction. So, I'm Sapta Banerjee. I'm an applied AI engineer at OpenAI. >> And I'm Tom Petrick, and I'm also an applied AI engineer at OpenAI. Super excited to be here. >> Yeah. So, we need to understand this concept as our whole session is based on the idea that a proposed fix from an agent is not always a verified fix, right?

`00:15` Proposed fix vs verified fix: why agent-written tests are not enough

So, suppose an AI agent that is proposing a security fix and tells you, "Hey, you know what? The code is updated and all my tests passed, right?" Now, this sounds reassuring, but before we trust the fix, we need to make sure that those tests are actually checked by an independent application. The agent might have blocked one way of triggering the problem, right? And written a test of that particular use case, but if there's a different way to solve that problem, that test case wouldn't pass, right?
So, we need to put that changed application in some kind of test to test it independently because to answer, you know, two or three questions like, "Okay, can we still carry the attack, right? And can someone still, you know, if the application is behaving current correctly even with that proposed fix by the agent?" These are the two questions which are very important to answer when you're trying to use an agent when you're trying to use an agent to, you know, do any kind of security fix within your application.
And that's what we mean, right? Like uh that's what we mean by a verified fix. It's like we tested it with we trust it, the attacks checked were blocked, and normal behavior still worked, right? And it doesn't mean we have ruled out every possible security issue, and it's very important to note that as we give agents more responsibility for security work, we need the connection between what they propose and what we can actually demonstrate within our workflow.
So, you know, the first thing you need to do is you need to understand the problem. That is very important, right? And like maybe you are security engineer in a security company, you investigate the problem, you will understand the issue. You write the actual code, the fix, right? And you test your application. And you produce a repair that someone can then review. Now, the first layer is your model reasoning, right?

`02:11` Three layers: model reasoning, Codex harness, Codex Security

So, suppose a report says that the application return a wrong address or a wrong report for a company. Right? The model actually kind of reasons is kind of understands okay, this what the problem is, right? It uses its intelligence to found okay, so this might be a, you know, a fix to that problem. And it can also identify any kind of missing checks or the security threat that has arise. But spotting a possible problem doesn't test it or change the application.
For that you need an environment. And that's where the second layer comes, where it's like the code of security, right? Or the code is the harness, right? So, harness engineering for you for everyone or not not familiar with, just let you know, it's like an environment to make sure like our models are sort of on it, like our state of the art, right? So, it kind of gives you an environment to make sure, you know, the model has all the tools, right?
The necessary permissions and everything to perform its action and it's performed in the best possible way. Now, the third layer, which is the very important layer, is actually the code of security, which has a specialist security workflow. So, these are like specialized skills which are given to the models in order to guide the investigation. So, these skills are specifically built for your cyber workflows, right? And, you know, it kind of helps you organize the kind of helps the model to organize the work around finding a possible vulnerability, checking the evidence, preparing a focused repair and experimenting the result.
And the aim is to give the reviewers all sorts of, you know, a line of code, you know, the uh the proposed fix around that line of code. Why are they doing this? Why are they building this line of code? In order to make sure, okay, this makes sense. We have tested it and we can move forward with the fix. Now, so these three layers actually work together. The model kind of reasons the problem. Codex provides the environment and tools to act on them reasoning.
And then Codex security kind of guides that work through security specific procedures, right? And this combination actually helps you build that security workflow. So, before we dive deep into a uh demo, because we'll be showing uh um like a like a end-to-end uh scenario, right? Where we will be mimicking to show like how an agent actually proposes a fix and how are we, you know, testing that workflow in an independent application like using Databricks OmniGen and Databricks app.

`04:28` Five-stage governed workflow for cyber agents

Uh we we want to kind of tell you like how are we going to move from capable agents to govern workflows, right? So, there are kind of five stages. Uh investigate The first stage is like investigate the report and source to identify a possible kind of vulnerability. Uh reproduce the attack using any kind of unchanged application. And then ask a person to authorize a narrowly scoped repair, right? And the order actually prevents uh finding from becoming permissions to kind of change the code.
And um you know, even if you have like an old test results or that has been reused by a new revision, you can actually use these kind of governed workflows to become successful. So, now, uh let's assign these responsibilities. So, Tom, over to you. >> I really want to showcase a simple vulnerability so we can focus on OmniGen as a meta harness coordinating this team of agents with very distinct roles and boundaries, as well as focusing on guardrails and human in the loop.

`05:19` Demo scenario: path traversal in a document portal

And we wanted to showcase from OpenAI our agents SDK, which we use to build our security investigator agent, the Codex harness, as well as the cyber capabilities of our models in general. So, for this application, you can think of it as a document portal where someone from company A should only be able to download the quarterly reports from uh their company as well as company B where they should only be able to download their company reports.
Um and as you can see from the synthetic ticket, um someone from company A was able to download and return company B's quarterly report. You can see the status has been escalated for investigation and there's no root cause. So, a little bit about this vulnerability, it's super simple. It's just a path reversal. So, you can think of this as an API request um a get documents where someone from company A has an authenticated session and they pass in the file name.
So, for example, it'd be quarterly report. txt. But, our application doesn't check the boundary, so someone is able to use path reversal while they're passing say dot dot slash to go to the parent directory. So, from company A to reports and then ultimately into company B's to download their quarterly report. Super simple, but uh you know, a major vulnerability. So, getting into the demo, as you can see here, we have the support ticket and then we have the actual reports that you can see here from company A and company B.

`06:47` Security investigator agent built with OpenAI Agents SDK

So, the first thing we're going to do is call our um security investigator agent, which is built using OpenAI's agent SDK. This has a very narrow scope. Its responsibility is to take in the support ticket passed by Omnigen and to go and investigate the application source code and come up with a hypothesis of the findings whether this is a vulnerability, um whether this is a permission issue, whether this is a false positive cuz right now all we have is this ticket that came in.
So, the first thing is we should be able to see that our security investigator is running. As you can see here, um so, first thing I'm going to do is show the actual evidence that it comes up with. So, um as you can see here, in our files, we have our evidence and you can see the findings. json. So, this will actually show, for example, the severity of the vulnerability and it actually backed in the evidence based on the source code.

`07:39` Independent Databricks verifier reproduces the attack

From here, we we our independent Databricks verifier. So, going back a little bit, um as soon as what the investigator determines that this is a vulnerability, we have a separate data bricks app that um runs and reproduces the vulnerability in a sandbox environment. So, here we have a model assisted plan and a testing suite to go and generate uh potential uh HTTP requests that will reproduce this vulnerability. So, here you have a separation of concerns from the model determining uh based on the source code whether there's a vulnerability and then actually reproducing it reproducing it against the same version of the application.
So, as you can see here, our data bricks verifier is running this independent check. And right here, as you can see, there's original findings and then there's going to be a reproduction. json in our evidence here. And again, you can see the actual HTTP request that went against the application. You can see the payload and the HTTP status. And you can see that our canary's exposed using different versions of path reversal.
So, think of it as, you know, full path reversal sending in uh dot dot slash core uh dot dot slash uh company B core layer port as well as passing in, you know, encoded versions and just different ways to expose this vulnerability. From here, uh we have a third agent, our third patch, that gets passed in all the evidence from the security investigator agent as well as all the evidence from the data bricks verifier. And now we can go and approve uh a scope change.
So, here, as you can see in omni omnigent, we have an approval uh approval card which just approves what is the scope of the fix. So, here we have what the vulnerability is and which files it impacts. So, as you can see here, we have app. py as well as the regression test that it impacts. And from here, we go and approve. We originally used a deliberately incorrect uh patch just so we can showcase the difference between a model behavior as and then the actual data bricks verifier.

`09:06` Human approval of a scoped patch in Omnigent

So, here, the vulnerability only patches direct path reversal. So, you you think of this as, again, dot dot slash quarterly B or company B quarterly report, but it doesn't check if, for example, it's encoded or uses sim link escape, just other ways to reproduce vulnerability. And from here, we actually have our Data Bricks verifier automatically run based on this patch. And as you can see, um it's able to come up with a counter example.

`09:45` Catching an incomplete patch with a counterexample

So, here you go. We have our actual get request with the query parameter. You can see the encoded version is able to still return an HTTP with quarterly B's quarterly report exposed. >> So, Tom, quick question. So, what we're seeing here is like uh the the security threat is like it's pulling the company B's report when it's about to pull in company A's report, right? >> Correct. >> Okay. >> Amazing. So, from here, um we actually have our change log with the actual candidate.
So, this is the first patch's changes, again, scoped to just app. py and the actual uh regression test. And for this incomplete and using Omnigen, again, you can see the actual diffs. Clearly, using a string comparison is probably not the best way to solve this vulnerability. Again, this was deliberate just to show an incomplete patch. From here, we're actually going to have Codex, uh Open AI's harness, go and fix the patch.
So, here we have Omnigen sending all of the session context, including the results from the security investigator, um the original Data Bricks verifier evidence as well as the incomplete Data Bricks verifier evidence with the counter examples. And from here, I'm just going to kind of skip through, but uh Codex will prepare this patch. And as you can see here, the Codex patcher is working, and um and after it's completed with the patch, we'll have the Data Bricks verifier again reproduce these HTTP requests against this fingerprint this version of the application.

`11:03` Codex patcher and verified fix

The Data Bricks verifier is running its independent check, and that it verifies that candidate two is verified, and I'll show you kind of the evidence here. Here's actual changes that I made to app. py. So, the original one was a string comparison. This one is actually checking that the document root is the same for the candidate as uh as company A. And then last but not least, we can see here the DataBricks verifier confirmed that all five past pass and all nine uh HTTP requests verified with 403 responses.

`11:54` MLflow tracing and evals for agent observability

And just something that I really want to focus on is we use MLflow for our observability. This is an open-source uh application created by DataBricks. And this allows us to go and look through our traces. So, as AI engineers at OpenAI, a lot of times we're working with companies who aren't willing to share, say their harness details or their application code or specifically their data. So, we'll ask for traces so we can go and look through what happened on each part of the turn.
So, especially for like multi-agent orchestration where we're working with a bunch of different um you know, there's a lot of different turns and this allows us to go and actually debug and look through each part. So, you know, maybe we think it's a model issue, but it actually turns out to be a tool call. We can look through every single turn, look at the amount of tokens that the model uses, and go and iterate and improve.
Another thing I really want to emphasize is the importance of evals. I think the fastest way to be able to develop in and our kind of fastest-moving customers have a really, really strong set of evals. So, you can think of this as kind of test-driven development where um you know, especially when we're pushing into production, we want to have an evaluation flywheel where we're constantly iterating and improving our evals.
So, that when we make these changes, when we use Omnigen and we make these changes to our harness, we can tell exactly um what has improved and what has regressed. And as we see these edge cases, we continuously uh we continuously add to our evaluations so that um so that we don't cause any new regressions. And then last but not least, I feel like it wouldn't be a complete demo without the actual GitHub PR. So, you know, we go from the synthetic ticket to the actual PR.
You can see you kind of our first candidate was rejected. Um it was only able to reject the direct parent reversal, but as you see here with the encoded parent reversal, um returns HTTP 200 with the canary exposed. You know, and in probably in a production environment, you wouldn't want to commit an incomplete patch, but for this example, I think it's really helpful to see the original commit and then our Codex uh candidate as well.
And you can see here all nine uh all nine HTTP HTTP HTTP requests returned a 403 with the canary um not exposed. And yeah, with that, Sapto, I'll uh pass it back to you. >> In a nutshell, what we're doing here is we have Omnigen, right? So, Omnigen is kind of like a like like a like platform where you kind of give agents a task and kind of follow its progress. So, think of it's like a like a control plane where you're running your agents, right?

`14:00` Architecture recap: Omnigent as control plane

And then we have a commanding agent which kind of orchestrates the workflow and as an investigator agent which kind of investigates the problem, right? And what happens is is they both run on Open AI Agents SDK, right? And we are also using the Codex harness where we are running the code where like the Codex harness is kind of using it to test your code, you test your application code as well as run any kind of simulated test and regression test as well.
Now, the thing is like from your from your Omni agent, right? Um we we we kind of So, we talked about this this uh uh workflow where it's not just, you know, the agent proposes a fix, a kind of security fix or something, anything like that, right? But we need to test independently, right? In an independent isolated uh container to make sure, okay, the that the problem can be reproduced. And if the problem is reproduced and the agent has proposed a fix, this is the way to fix it.
That's where Databricks come in. So, we are of doing it within a Databricks Databricks kind of isolated container or application. And, you know, the the how the communication flows from Omnigen to Databricks is through HTTP request like an OAuth protected verify app, right? And then we kind of run a restricted runner plus we kind of simulate those HTTP attacks to make sure, okay, this makes sense, right? This is the fix to fix the problem, the security problem.
And then at the end of the day you have a human to actually go through that workflow, understand, okay, this is the proposed fix, kind of verifies it, and kind of fixes it like, you know, approves the patch, right? So, that's like the human approval. So, with that being said, I think we are at the end of our presentation today. Uh thank you for your patience. Uh we really really like the crowd. I hope you you enjoyed the demo and everything.
Um so, you can always reach out reach out to us over LinkedIn. So, me and Tom, we are very active in social media. Uh if you have any questions around emails, you know, ChatGPT, um using our models, Codex harness, and as well as like the concept that we talked about, feel free to drop us a DM. Uh we will be happy to assist you. Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.