Building Security Agents: How Cursor Blocks Vulnerabilities in Every Pull Request

Summary

Agent Orchestration: Teams must shift focus from tool adoption to operational maturity by establishing centralized governance and deliberate caching strategies.
Cognitive Load Management: Limiting active development environments to three or four tools prevents decision fatigue and maintains productivity as autonomous agents scale.
Human Code Review: All automated code generation requires mandatory developer verification to secure supply chain integrity and manage token consumption costs.

Building Security Agents: How Cursor Blocks Vulnerabilities in Every Pull Request

Talk by Travis, Security, Cursor
Cursor's security team built a security agent that reviews every pull request and blocks merges that have unresolved vulnerabilities, using specialized reviewer and triage agents that run in parallel. This video shows how to build security agents for code review that are fast, scalable, dependable and developer friendly.
The speaker traces the path from a simple GitHub Action running a "find vulnerabilities" prompt to Cursor Automations, then to a rebuilt architecture after runs exceeded eight minutes. Reviewers work asynchronously on the diff, fast LLMs deduplicate findings, triage agents decide what gets reported, and a lightweight MCP built on API Gateway, Lambda and DynamoDB sets the gate status. Precomputed facts give agents attack-chain context, developer replies feed false-positive heat maps, and the gate pages a manager unless an issue is acknowledged. The talk also covers token cost control, including a $100,000 token spend day.

Chapters

FAQs

How do teams prevent cognitive overload when deploying multiple autonomous agents?

Limit active development environments to three or four tools to prevent decision fatigue and productivity loss. This video discusses how enterprises manage tool sprawl while maintaining developer velocity as autonomous agents scale across teams.

What's the role of human code review in autonomous agent governance?

Human developers must review and sign off on all agent-generated code to secure the software supply chain. This video explains why automated code presents severe security risks and how to enforce mandatory verification workflows.

How should enterprises manage token consumption costs with autonomous agents?

Implement centralized caching strategies and deliberate orchestration approaches to optimize token usage. This video covers cost management and performance tuning techniques used in large-scale enterprise deployments.

What is agent orchestration and why does it matter for developers?

Agent orchestration provides centralized control and coordination of multiple autonomous systems. This video explains structured approaches to managing agent fleets without stifling developer velocity or creating operational friction.

Full transcript

`00:00` Introduction: building a security agent at Cursor

Hi everybody. Good evening and I'm Travis. Uh I have been through an interesting journey to get to this point. So March of 2025 I was actually running a startup with all the excitement that that entails. And then in June I joined Cursor which at the time it was like pretty obvious to me that it was going to be a good company. But in uh in about a year after that then we have been acquired by SpaceX. So I'm maybe I put the wrong logo on the slides.
I don't actually know like who I am or where I where I am anymore. But what I want to talk to you tonight about is what how we built our security agent and um the properties that it has and kind of like how we got to that point. So just about me uh I started a company called Resourcely in 2022 and before that I was actually a data bricks and uh like I mentioned the company that I started was not successful but I learned a ton.
I do not regret any of it. And at the time that I basically was complaining to my investor that I should probably go get a job again. Cursor was saying that they needed to ramp up their security. And so it was kind of like a easy match here. The the important part about this is that the purpose of the company that I built is to improve developer velocity and improve security. I've always felt that the trade-off between those two is bad and leads to bad outcomes.
And I've really been interested in like how can you have your cake and eat it too. So when I joined uh we had one security person before me um Henry. as employee number 20 which is actually a pretty good time for a company to hire a full-time security person. And the the company itself had done many things well. So they had set up you know octa from the beginning and like one password and terraform and there was a lot of good stuff going for it but I got there and at the same time we had dependabot and dependabot's like yeah bad news you have 3500 vols I was like okay well first of all it's a lot like we have to figure out how to do something about that um build a process build you know like v management trim down all the noise out of there one of the things that I decided not to do is do anything for SAS like static analysis the reason for that is not that it's not important, it's actually extremely important.

`02:09` Why agent-driven static analysis beats legacy tools

But when you look at traditional static analysis tools, I realized that like there's no way that this is not going to be agent- driven. And so I decided like instead of adopt something that's legacy and then try and like move it to agents when those get good enough, I should just wait, do other projects and then get an agent driven like static analysis code reviewer at the time. So what this gives you, I mean really the power of agents is that you know now they reason, right?
So instead, it's not like here's a giant list of rules. I have to make the rules. I have to test them. I tell the agent go find vulnerabilities. Like there's there's more to it than that, but like that's the root of it. And it can go and get creative, find them, and it can read through files like much faster than a than a human would be able to. So when I started thinking like, okay, models have gotten good. Uh we should probably like actually do one of these.

`02:59` Requirements: fast, scalable, dependable, continuously learning

I had some requirements. So first of all, cursor has the highest um development velocity of anywhere I've ever seen. and like higher than I could have imagined. So the the rate at like for me myself like I'm a security person and I'm think I'm averaging like 20 merged PRs per day like somewhere in that neighborhood. And then there's like there's people in my company that are like much much much more than that. I'm in like actually like 20th or 30th percentile for token usage.
So it needs to be fast. Like anything where it's like hey sorry you know bad news like Travis is here now and like all of your PRs are going to be slow and crappy is probably not going to go over well. Um, and it needs to scale. So like with that velocity, if it's like, you know, it's a it's a system, but as soon as the PRs, you know, go through the roof and then it starts like falling over, that's not going to work well.
Um, it needs to be dependable. So my ultimate goal here was to actually like prevent vulnerabilities. This means that we're not going to do like, hey, here's here's like a nice suggestion, like why don't you think about it? Uh, my goal was to actually like have this be a hard block. And in fact, it got to that point. It is it is the only hard block at the company. So if you want to go ahead and merge code and it has an active security issue that you haven't acknowledged, you have to either reference an active incident or page your manager.
And it's not like, you know, go please page your manager. It's like my my system will page your manager. Like your your manager is going to be paged. So this thing needs to be reliable. It can't be like, oh, I have to page my manager because Travis's thing fell over. um we have uh a lot of kind of like ideas about the way our application like works you know should work our threat model but these things are always changing and so I didn't want to just write you know one set of rules and have that be it forever I wanted the system to continuously learn um and very de developer friendly so it's not like you know here's a finding you have to go click this link like log in do all this stuff like we want to put these things like right there you know in GitHub or source control like wherever like the actual action of the change is happening like it needs to be right there.
So that was what we wanted um out of it and I started you know the October I was like okay now's the time like I've done enough other things and agents are quite good now let's do one of these and so in typical kind of like cursor MVP iterate fashion I just threw up a crappy one uh I made a GitHub action that triggers on pull requests and it's literally just our CLI wrapper and the first prompt was basically like find vulnerabilities um and you know it did like it found vulnerabilities now many of those vulnerabilities did not meet the requirements I said earlier like I wouldn't be comfortable putting those in front of developers and in fact I think up until like mid uh January I didn't show any of this to anybody like it was just like I'm telling it find vulnerabilities I'm refining the approach and then finally got to the point where it's like okay let me show this to you know developers we're not blocking yet but let me show them and actually like by then it had gotten so good that developers did something like very odd from my perspective in security they said this is great like we want this like front and So I was like, "Okay, good."

`05:03` From a GitHub Action prompt to Cursor Automations

Like we have, you know, what we'd call like internal product market fit for this thing. Um, but you know, this is like some kind of like rickety GitHub action trigger that I'm maintaining. So the next break here came in December. I don't know who's heard of uh cursor automations. Anybody use those? Just just you. All right. Well, thank you for that. Um, you should check them out. They're they're very cool. But basically what this gives me is instead of me having to maintain like a GitHub action trigger automation says on you know any of these kind of triggers like we're going to go run an agent with a prompt for you and it can have like connected tools.
So like that took a lot of the not security agent parts out of this for me so I could just focus on making my things good. And so I put that out and it was a refined version of my prompt like find security vulnerabilities. By this point, we had done more effort to like tell it what we mean by security vulnerabilities. And we'd learned things like, you know, agents might make stuff up. So, we have to push them to do extra work.
You know, basically like prove the attack chain, explain the impact of this, like you cannot report anything until you have validated like line by line, you know, file by file like how this vulnerability actually works. But it was good enough then that then uh people stepped up from my team and said, "Oh, what about like infrastructure as code? Can we make one that's specific for that? Can we make one for um like weird cursorisms?

`07:07` Specialized security agents and the pull request gate

You know, we have some like kind of like pave road library kind of things. Can we make agents that look for that? And so now it's like we have specialization of agents. And then by February it's like okay, it's gate time. Like finally like I'm meeting my goal. I'm going to block on anything that's not acknowledged. So the way this actually works the agents run, they put up comments. You can reply to the comments. So then this thing was like running and we continue to scale.
And then finally what happened is it got over eight minutes. like there was enough of these things running like they were getting clogged up and it was way too slow and so we completely rebuilt this architecture um was something that I'll get into more. So we want to have basically it's like every single PR like you think about the challenge that security teams have right like you you have a set you know a finite amount of people like maybe you're doing threat modeling maybe you do code review you're kind of like battlefield triage like oh this thing's like not that important of a system we're not going to review it you know this is like seems like a small change we're not going to review it anytime that you decide that you might need to look at something your cognitive like overhead of switching into that like taking a look and getting out and so like companies have always had to get these like weird heristics about like whether they are or aren't going to get involved in a change.

`07:33` Rebuilding the architecture when runs took over eight minutes

And so like the the really cool part about this is like now we have these like agents that as long as you feed them tokens, they can go cover everything, which is black magic if you would have said it 5 years ago. Like nobody would believe that this is possible. Um and so yeah, this is what I said, right? Like the it's a gate. It's hard required. Uh you're not going to pass, you know, outside of those like pager manager or whatever.
Um it's fast. like this is faster than like my goal was like this is never going to be the slowest thing in CI like ever. Um and then we're reporting high impact things. So we I don't like you know kind of like stop light colors for risk. I think it's kind of bad but actually the models like really like it like they were trained on it. So if I push it to like quantify the impact in stoplight colors that actually ended up being higher signal which kind of surprised me.
And so we're showing developers like actionable findings that in a fast way and then we're blocking on it. And practically what this means is like most of the developers actually like don't read these either. This is for their agent. So like they work on a PR, they put it up uh they have these commit statuses, their agent is in like babysit mode like watch CI and then like make any changes and then it gets feedback back on security things. that model at that point is like most of the time like you're right and we get we have analytics on all of this like most of the time model's like yep good catch I'll fix that and it does and then the cool thing is we can actually like mark that as fixed so now we have like quantified value that this thing provides like all of those mark fix issues I'll get into are validated by the LLM that um this thing has been fixed since the last commit when we marked it.
So this is kind of like how the the architecture works here. I'm not gonna like piece by piece like read all of this, but it's basically what I said like we're running these things on build kite. Um, we have this package which for us is called ASR local. It runs off the latest main. We can also like pin it to some something else if we want to test like some change to it. It goes and like figures out, you know, for a given diff like what are the different like file paths based on that like what kind of agents need to get involved?

`09:47` Async reviewer and triage agents with deduplication

What what's cool about this is actually like it's all async. So we have all these different reviewers. they're all operating on the diff at the same time. That's part of why it's fast. And then as soon as one produces a finding, it drops it into a like basically like a queue um in the agent package. And then the the script that runs it is looking for those things, it will call another LLM, like a fast LLM to do a quick ddup and make sure that it's not the same as like something else that's already being triaged.
And then if it's not, then a a triage agent picks it up. And these are like different kind of like personalities. So review is like broad and fast. It's just like find anything that might be an issue. Um, it's got like a tiny bit of validation just to make sure it's not like pure garbage and we're wasting triage time, but triage agent is the one where it's like you have one issue. Your only job is to like figure out if this meets our criteria and should be reported or not.
Um, and then we do another ddup. So all of the things that like triage, you know, in case we like missed it the first time or like triage decides that actually like these two are the same issue, there's another dupe fast llm and then I built an MCP for this. So like the way that the agents actually like do all of this stuff is with this like MCP actually it's just like an API gateway lambda Dynamo storage. So it's like very very simple.
The whole thing serverless from that perspective um and then it has the ability to like set the gate status um go and like pull comments things like that. So I mentioned this but yeah these are the two personas. This is actually like one of the breakthroughs that we had between like when we started and this point is to split them out this way because it it sets them up for like roughly equal level of work. And we really want one to be like creative and we want one to be analytical.
Um and in doing so they're roughly going to get the same amount of work. They're going to roughly finish in the same amount of time. Um and each reviewer like has its own prompt. So like the one that's looking for like cloud mis configuration like infrastructure as code its reviewer prompt and its triage prompt are going to be different than the one that looks for like are you using our privacy mode like paved road basically.
So yeah reviewer it's looking at you know this like full diff. It's going to like one other unlock that we got uh that was not obvious from the beginning is a lot of the stuff that like all of these agents need to know is actually going to be like uh let's say that you're you're doing a review and you hit like this path you're like this you know input can be like attacker controlled and then the question is like what happens downstream of that like is it sanitized does it go to a thing where like it doesn't matter if it's controlled so like all of those are kind of like paths in your attack chain and it doesn't make sense for each you know review or triage to like be doing all of that work in parallel.

`12:18` Precomputed facts for attack-chain context

So what we do is we premputee faxs and we have a an automation that runs a few times a day and it will just go and harvest new facts or for the existing facts it will go and like you know because line numbers move and things like that. So we'll go like update the locations of the facts and then we don't want to overload like the context of these things with every fact discovered everywhere. So basically we do like a fuzzy match on the fact and like where it is and decide which one of those should be fed to whichever one of these agents needs it.
So this is kind of an example of how it works. But yeah, so like we have these different paths. Um based on like what the path is, we might get one or more different like reviewers into it and then different like triage modules selected for it. So it's really like we the goal is to avoid doing extra work. Um in part because we want it to be fast, in part because if you tell an agent to do too much stuff then it kind of gets worse at everything.

`13:25` Controlling token cost and agent runtime

And then also because of token costs. So when I was really frustrated with the runtime, I was like, I'm just going to do everything possible. I'm going like rearchitect this thing. Carson and on my team like just went and spiked on this. We were like, we're just going to get this thing done like this week or die trying. Um, but like one of the things that I did was like, I'm going to upgrade the model that we were using.
It was like GPT53. I was like, I'm just going to put it on GPT55 high like all of these things. Anyway, I did a $100,000 token spend day. So, I went and looked at it. I was like, "Yeah, that's probably a little more expensive than we need." So, anyway, this is like dialed back to I think it's like 54 medium fast. So, like a triager might um in the process of like going and digging like whatever it's doing, it might say like, "You know what?
I've discovered a new fact." And so, it can use its MCP to actually like propose that up. Um and then that gets like reviewed by another agent that runs multiple times a day and decides whether it like meets the bar for a triage fact. Like we actually the triage one is not like I need to be very thorough. It's just like a maybe candidate fact and then another one will review it. We want to harvest what our users are telling us.
So if you know a lot of uh when I started looking at these things it's like oh that's a false positive actually like false positive is kind of an overloaded term. So there could be a few ways you get a false positive like agent is wrong. It like you know made up some path and like this is what the kind of like validation is meant to solve. Um, but most of the time it's actually the agent found like a legit security issue and it is like widely known in the company that this should be out of our threat model.

`14:44` Learning from developer feedback and false positives

Like it's not something that we care about. Like this is just a like risk accepted thing. And so like it's considered noise. Anyway, by harvesting all of the comments from our users, we get like heat maps on that really fast, you know, like um a lot of times like we get like agent says like, "Oh, good spot. I fixed it in this commit." But every once in a while we get like a you know, stupid security bot. And like when we get five of those and two of those, you know, on something that's like the same class of issue, then we're like, okay, this is probably like we should take a look and decide like this is below our cut line or not.
So yeah, this is about the MCP. It has these paths, whatever. Like these slides will be online. You don't have to read all this. Now, I think the general purpose of this is that the agents want to be dealing with tools. And so what I've done is like created this like very lightweight serverless bridge where it has all of the tools and then we have something that's like reliable that is a persistence layer and has something that is not just giving the agent like the GitHub creds where it can like set gate statuses.
So tools are really nice when you want like a well- definfined boundary between like an agent and some other system. So we made one of those. Uh so the yeah human replies to the to the comments is actually like extremely useful. And so if you I I think I it was like a triquarter or something. There was a system a long time ago that I think Google had or like something about Google, but it was like you'd have emojis or something like thumbs up, thumbs down or something like that and then if the like thumbs up to thumbs down ratio got below 80% the system would just like pull the rule out and it would go back to like whoever created it.
It's like hey this is like too noisy. We don't want it. And so now in this system we actually get like better than that. It's not like oh this whole rule sucks. It's like this this PR wasn't good and we actually like know what the finding was to it so we can actually like get very very good signal on it and this thing just kind of like runs by itself all the time. So like I as the person who like maybe like put in this like oh and by the way look for this I don't have to do anything like an agent's going to come back and it's going to be like um based on the developers feedback like this thing seems to be like not something that we generally care about.
Is that right? Be like you're up and then it'll just like mark it out as like a like a fact or like a qualifier in the triage agent. There's like a bunch of ways to skin this cat. Um, the gate. So, I I don't know like how into gates y'all are going to be, but um I'm really into it because it seems like the kind of like most developer unfriendly thing that you can do, but actually like what it does is it um introduces like just enough friction that like people like usually their agents will just actually take it successfully.

`17:03` Why a blocking gate works with coding agents

It's like I can't merge this thing until I do block. If it's not like a blocking gate, it's like okay cool information like I don't need to do anything with this. But like blocking gate like the agent is actually like a very good um at accomplishing the goal that you give it. And so it'll be like okay like what's up with these security issues. I should actually like probably do something with these. And most of the time it's not like a throw a BS comment.
It's actually like yeah this is legit, you know, thanks for finding it. And it'll just like address in a loop. So we're really into this um idea of like the software factory, right? like we want to get to a point where kind of like all of the developers are like product managers and they're controlling like a swarm of agents and those are your SWES and then the agents have like exit criteria. So like their exit criteria is like you've created tests all the tests pass you know you've actually like if you have the ability like spun up the environment so we have a whole harness where we can spin up like our client or our backend or whatever like actually verify the thing end produce proof of that and all the gates pass.
So this is our like security like part of the factory basically. The last point that I wanted to make here is a cool thing about cursor and I I touched on this a bit earlier but um we we have like a virtuous cycle where like the work that we do on the security team will either um like we will be a heavy dog foodter of products that are coming out or the other way around. So like this is um an example of both. So, like when the automations thing came out, um the engineers that built it came straight to me and they're like, "Hey, we know that you're doing this like janky GitHub thing."

`18:31` Agent Kit and the product feedback loop

Like you actually probably like really like this product. I don't know this, but maybe they knew that we were doing it anyway. And like that gave them the idea, but from my perspective, they're like, "We think you'd like this. Like, go use it." I used it. I found a bunch of bugs. I complained to them nicely and then they fixed those things and I'm like, "Yeah, this is great." And then we became like the heaviest user.
Um so much to the point that when I was like, "Okay, this is too slow now." And when we we moved off like their like total revenue like we have our internal you know we have like total revenue and then external revenue. Their total revenue like took a giant dip. Um I had told them but like also like somebody noticed on their chart and they're like what happened? So um this is really cool because we have the ability to like get early access things and then use them for cool security stuff and then the other way around.
So like when I actually moved this to the new architecture um I told them why I did it. They were like you know maybe we can fix these things. I'm like, I I would love that, but um but like I need this to be very fast. Like we're we're failing open our gate. That's vulnerability. So like please fix it. And then meanwhile I'm going to this. And then they took the idea that like I moved to this and they're like actually like you know what Travis is trying to do is probably something like pretty common.
Like you do want this like hybrid of you know deterministic script and agent. You want to be able to like blend them. And so the team has created um something called agent kit which I think is really cool. and allows you to like compose these kind of like agent script uh hybrid things in a way that would make it really easy for you to do today like what I'm mentioning. You wouldn't have to go through all the work that I did.
So, I really like that bit. It's kind of like a virtuous cycle that we have with the product team. Um which makes it quite fun. And that's it.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.