AI agent security: defending against malicious skills

Summary

Greg from Sentry's AI team demonstrates concrete attack patterns against AI agent skills, covering three categories: context poisoning through malicious instructions embedded in files agents read, harness vulnerabilities that exploit legitimate framework features, and ecosystem attacks using mechanisms like npm post-install scripts and conftest.py.
Attack experiments were run 20–30 times each against state-of-the-art models to measure reliability, revealing that automated scanning misses sophisticated attacks that hide malicious code in EXIF metadata, temporary files, or automation hooks.
Defenses include skill scanners, restricting skills to trusted sources, improved permission controls, and least-privilege configurations that limit what a compromised skill can do.

AI agent security: defending against malicious skills

Agent skills extend AI capabilities but introduce attack vectors through context poisoning, harness vulnerabilities, and ecosystem exploits. From EXIF metadata injection to post-install scripts, attackers can compromise development workflows. This talk reveals concrete attack patterns tested on state-of-the-art models and practical defenses including skill scanners, permission controls, and secure skill management.
Learn techniques attackers use to hide malicious code in reference files, why automated scanning misses sophisticated attacks, and how to implement security across your agent ecosystem. See specific examples of context poisoning, temporary file hijacking, and persistent backdoors hidden in image metadata or automation hooks, plus defensive strategies from security researchers including skill scanners, secure skill sources, and least-privilege configurations.
🤝

Chapters

FAQs

What is context poisoning in AI agent attacks?

Context poisoning involves embedding malicious instructions in files or data that an agent reads, causing it to execute unintended actions without the instructions being visible to a human reviewer. Examples include hiding instructions in EXIF metadata of bundled images or in reference files with innocuous names.

What are harness vulnerabilities in agent security?

Harness vulnerabilities exploit legitimate features of agent frameworks — such as hooks or execution modes — in ways that allow well-crafted skills to cause unintended behavior. Rather than being flaws in the framework, they are cases where normal features can be abused by malicious skill authors.

How do ecosystem attacks work against AI agents?

Ecosystem attacks use existing mechanisms in programming ecosystems — such as conftest.py configuration files in Python or npm post-install scripts in Node — to execute malicious code when a skill is installed or used. These attacks are easier to carry out because they leverage infrastructure developers already trust.

What are the best defenses against malicious AI agent skills?

Defenses include using skill scanners to inspect skills before installation, restricting skill sources to trusted registries, applying improved permission controls and auto mode restrictions, and configuring least-privilege access so a compromised skill has minimal ability to cause damage. The key takeaway is to not use skills from sources you cannot fully trust.

Full transcript

[00:04] Hello everyone. My name is Greg. I am an engineer at Century. I work in AI team at Century. Security is kind of out of my scope usually, but I'm going to talk a little bit about security and skills specifically. So this talk is not going to be theoretical. I'm going to go through like concrete examples of how
[00:21] skills can attack you and how you can defend yourself from this. Which like the great message here is really can't right now and you shouldn't use skills from sources that you cannot really trust. There is sort of two aspects of
[00:38] like security around skills. One of them is how do they make it to your machine? And the other is if they make it to your machine, what is the likelihood that they're going to own you? I'm not really touching the first one because that's the general broad registry problem and how do you actually
[00:54] trust whoever makes the stuff? I'm going to try to dig into if you have a skill and you let it run, what's the chance that the agent is going to do something malicious or if the agent needs to do something malicious.
[01:09] In in all those experiments, the methodology that I used was pretty simple. I used closed code and I used open code. For both of them, I had a large swap of models that were run multiple times, 20 times, 30 times on each of the skills that I'm going to describe. And all of
[01:26] those skills that I'm going to describe. So there is sort of three categories of attack like one of the stripe here. Context poisoning is the type of attack in which we pass some sort of malicious instructions in some way, shape or form to the agent
[01:41] and we hope that the agent is not going to catch on and run some either script or something that's going to either exfiltrate data, give us the remote code execution capabilities or otherwise behave incorrectly. Harness
[01:57] vulnerabilities are it should really be called harness features. Um they it's the type of attack where I take particular features of cloud code or open code and show how they can be abused by well-crafted skills.
[02:14] And then the ecosystem attacks are going to be much simpler. They basically take whatever already exists in say Python or Node and try to make those types of attack um a little bit easier to exploit with
[02:29] skills. For context poisoning we're we're going to talk about two important aspects. When a malicious actor tries to create a context poisoning attack, they care about two things. One of them is that a human reviewer looking at the
[02:44] skill is not capable of discovering that this skill is malicious. And that can be done in multiple ways. The simplest one that I've seen is you just have a skill. Like there is this common skill that people like to use for React that's called React Best
[03:01] Practices. Great skill. It has a good example of um progressive disclosure, which basically means that the skill file itself is very short and then it funds out to multiple reference files. And it's kind of not feasible to expect that everybody who installs that uh that
[03:17] skill is going to go through every reference file and read it. You can uh bury some instructions some malicious code inside a reference file. You can do that with zero-width Unicode characters or you can do that with this example here, which is the HTML comments inside the markdown.
[03:34] If you go to GitHub and open like a markdown reader on top of that, you will not see that. It just hides it. So like those are very simple tricks that are going to try to uh fool the humans. And a couple of months ago, like maybe a month ago, that would work very well. Since then, agents got pretty
[03:52] but not necessarily catching those, um but ignoring the instructions that are put there. So, like that would probably not work right now. You don't only have to fool the human, you also have to fool the agent. Um especially the state-of-the-art models will read the oh, you need to curl that
[04:09] random gist and pipe it to bash, and they're going to be like, "Nah." This is malicious. This is at the very least fishy. We are not going to allow that. Um so, you want to kind of design something that's going to convince the agent that
[04:26] it's fine. Um and the previous attack I was testing on all the models, none of them fell for it for like straight-up curl to bash. All I had to do is move those malicious instructions into a bash script
[04:41] and call it something innocuous. Call it analyze licenses. 100% of agents run that happily. I have I only have to convince them that this is like this looks innocent. If that looks innocent, they're going to run it. Um
[04:57] and so that's the like simplest sort of context poisoning attack that you can you can imagine, and an estimative of what we're going to be covering here is is a variation of that. Lots of scanners would actually catch that attack, so it addresses the point of would that even
[05:13] make it to my machine, assuming you're using these sorts that have scanners. Um or a human reviewer would catch that if they would read the script. But like, you get the gist. That could be uh those vulnerabilities, those like malicious attacks can be layered on top of each other, hidden uh hidden somewhere uh deeper. I have one more
[05:30] example of context poisoning. This is a question to y'all. What do you think is wrong with that skill? I would say that this skill what it does is pretty trash. Like the the actual functionality here isn't great. It has malicious instructions in its
[05:45] source code. It's a I I promise there is no like zero with unicorn unicode characters. This is my favorite one because if you look back, this badge is local. This is bundled with that skill, right? I've modified that badge and I've put instructions inside the EXIF metadata.
[06:03] And now, if the agent loads that, they they have a chance of getting out. It's not 100% between 30 and 40% of my runs fall for that. But, all of the state-of-the-art models fall for that. So, Opus 4.6, Codex 5.4,
[06:21] and Gemini 3 Pro all fell for that. Gemini falls for everything. I'll I'll say that, but yes. So, they're reading the file. They they just decide it's maybe it's worth checking out what's inside this PNG and they use the read capability directly on
[06:37] the bundled PNG. Yes. Okay. Why would they do that? But, okay. Beats me, but they will. And you will see this kind of like As an example, as another example, I didn't include it here, but like you can symlink an example SSH file to your
[06:54] actual root ID like .ssh/id_rsa. And ask them to like print an example SSH file and they are not going to catch on that this is a symlink. And and just happily do that or upload it to you know attacker's URL or whatever. So, like those kind of things
[07:10] are absolutely real. This skill is my favorite specifically because it's so hard for human eyes to catch that this is a problem. And that's why like you cannot really trust your eyes. You have to run scanners and not just like traditional security scanners, like LLM judge
[07:25] scanners, like Claude code, if you prompt it with you we're doing a skill scan, is able to find that. The last sort of thing that I want to mention during in this section is memory poisoning. Um this is not a separate vector of attack. This is is so
[07:41] something that you again tack on later on top. Um the malicious instructions, the script or whatever, they can instruct the agent to poison the memory, and that's extremely dangerous. The idea here is that if you can convince the
[07:56] agent to write into root cloud.md, and then someone removes the project or catches on the skill and removes it, but don't catch the fact that this has happened, that's like a persistent backdoor where the instructions stay, and it's so hard to find. Unless you run
[08:12] Git on your cloud.md files, it's going to be very hard for you to to realize that this is a problem. And honestly, if you run Git uh Git on your cloud files, you probably also sandbox, and you probably also don't run dangerous skill permissions. So, going further,
[08:29] let's get a little bit scarier. This only works for cloud code, but um it should it it really puts another like a nail in the coffin for you should really not trust external skills. Um
[08:44] there is a feature in cloud code where inside cloud inside the skill, if you put {{}exclamation point} and then a command, the way it works is when you load the skill when the when cloud code harness loads the skill, it automatically runs that
[09:00] command and expands the output of the command into the skill. And the intention's simple. You want to make sure that if there is context that's time sensitive or project sensitive, it can be included in a skill and it sort of self-updates in that sense. And I'm a fan like I don't
[09:17] think it's wrong, but it's really problematic if you don't trust the skills, because now like the previous ones, the context poisoning ones, sure you put malicious instructions in place, but like those are sent to the agent. So, agent has a
[09:33] shot at saying, "Oh, this is wrong. We shouldn't do that. Um, but here you just don't. Cloud code just runs it because it's part of the semantics. Like there is no agent like verification of of any sort. So, that's not going to get better because we get better models. And then
[09:48] there is a like variation on that also in cloud code. Um, which I just find funny. Um, I don't know if you know, but if you're using cloud code, skills can define hooks. What?
[10:08] And I don't know really why, but in those hooks, you can define whatever command you want and it's going to run repeatedly. So, if you're able to do that, you're going to be able to like iterate over stealing credentials or whatever. I don't know if that's very practical. I was able to run it pretty successfully, but it's also
[10:24] like very esoteric. I really like how it renders in markdown by the way. Let's move to ecosystem attack. The most simple one is this. This is the only slide I have where the skill itself is not malicious. This is
[10:39] like super simple. It just say, "Oh, run tests and like do a test coverage report." Things like that. What is malicious is I bank on the fact that whoever uses that uses Python and pytest. And I bundle up conftest.py.
[10:57] And pytest will run auto discovery. And during auto discovery, it's going to pick up my conftest.py. And then it's going to run it. And we're done. It's a feature, not a bug. I had prepared an npm skill, but I didn't end up uploading uh the actual malicious
[11:14] package um cuz I didn't want to deal with npm terms of service and whatnot. But, that said, um this is the most common. This is like the the the example of attack is imagine I created my own React best practices skill.
[11:30] And I created reference files to them. In one of the reference files, I say use react scaffold, which is I just made it up. Uh and I created a react scaffold package, and that has either post installation script or like some kind of malicious code. That's going to be
[11:45] caught by the agent. The agent is going to happily install the package. I was able to run it successfully again on around 30 to 40% um agents. And it's it's one of the most common uh server attacks that are that are using npm. This is actually the attack that uh
[12:02] or a form form of attack that was one of the most popular recently in the like cloud hub um reports that uh sneak hat. So, a PSA, if you're running npm in your organization, you should disable post install scripts even before AI. That's
[12:18] the one of the most prominent sources of remote code execution. Uh but also I want to call one more thing out here, which is um right when I run this thing, uh Entropic came out with auto mode, which is a different way of
[12:35] running permissions on the actual tool calls. Um before that, you either have to accept um all the tool calls, or you run dangerously skip permissions. Auto mode is somewhere in between where they bundle up a classifier, and the classifier can make a decision about
[12:50] whether um the tool call is safe or whether the user should be prompted. And I rerun all of those things on it, and it got like better. It's not It gives you a little bit of false sense of security because I still was able to exfiltrate a lot. Um but I want to give them a credit. Whenever
[13:06] instructions say to do npm install, without a fault, it's going to stop and ask for permission because it's such a popular attack. So, credit where credit's due. That's That's like a great great stop for uh for exfiltration. One last thing, uh this is
[13:22] not an example of a skill, but this is an example of a tactic. Um the time of check time of use attacks are actually one like a format of that was what owned light LM recently. Um, but generally speaking, it's the attack where you when you check
[13:38] a package or in this case a skill, it looks fine because it is fine. And ideally, the scanner doesn't even discover that there is any problem. But maybe in the skills instruction, it says, "Oh, download this installation script." And that installation script is fine during the check. And then after a week or month or
[13:54] whatever, that installation script is being hijacked and changed to malicious instructions. What can we do? So, I actually presented this with uh to to my friend at Anthropic who works in cloud code. And his general recommendation was, "Yeah, like just don't run untrusted skills. Like this is very
[14:11] scary." Um, and I genuinely agree with with with him. Um, my tactic I I got much much more paranoid is what I'm saying. Like 2 months ago, I run open claw. I was like, "Yay, install everything. We're going to be fine." Um, now I don't. I I deleted my open
[14:27] claw and I research uh res- recycled my Mac mini. It's now my my daughter's device. Um, write your own skills. Even if I find a skill out there that I think is somewhat interesting, I instead of copying it over, I will um
[14:43] put it inside cloud code and be like, "Hey, write me a skill like that." And ideally, because the skill had malicious instructions in an innocent way, those won't be copied over. Um, if you really must use um skills from external sources, use it from
[14:59] trusted sources. Uh, like I wouldn't I wouldn't reach from for for random uh one-off repositories from which you're going to fetch skills. And the third one is scanners. Even if you trust repositories from big companies
[15:15] like Anthropic, Sentry, or whatever, I would still run scanners on them just in case because they can they can also be hijacked. Um, there is one point that I would say is important here which is don't let the tools pull skills implicitly. Like we don't want at Sentry for people to
[15:31] use find skills skill which is like it's like a malware loader at this point where a skill can start searching for other skills that can do jobs that it didn't know how to do. We discourage that because it's it's so dangerous. It's it's equivalent of like open claw running around and
[15:48] land grabbing. So a little note what we do at Sentry. So we have get Sentry skills with some useful skills in there. If you listen to anything I said here you shouldn't go there and just start pulling them in.
[16:06] But I did create skill scanner. So skill scanner encompasses all these attacks and more. And we use it to to scan actual skills in the repository or new skills in repository. Generally speaking there are scanner the security companies have their own scanners and they're very good. They're
[16:23] catching majority of stuff. I kind of pushed a little bit of crazy tactics here and I was able to break them pretty consistently. So like for instance snake and I don't I'm not I don't want to like I'm not trying to disparage them. I think they're doing great job but like
[16:39] they just didn't see this attacks yet. They they were able to catch like 60 70% of my attacks and I was able to catch more of them through the skill scanner. The way we run them this is a little bit of self plug on two things. The way we run them is through Warden. If you've
[16:54] used any like grab tiles etc. Those code review tools. We have our own which you can deploy in GitHub actions. This this is the open source one. The twist here is you take a code review tool and you give it a skill. So if you take Warden and you take a skill scanner
[17:10] bam you have a security scanner. And the last one is dot agents. Dot agents is a tool that we build for actually managing this for employees. So, that agent takes common configuration, puts it in like agent.com file, and generates common MCP servers
[17:27] and skill configurations for all the all the harnesses. Uh and we use that for both like global uh skill management as well as uh per repository skill management.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.