Skip to content

AI and CVEs, a Contrarian View

I am going to say some things that will likely feel very contrarian to many of the people in information security. There are a lot of people who believe that AI is the next harbinger of CVE exploitation. There are others who are heavily compensated by selling that fear. Both keep saying things about AI-bred exploitation like "it will" happen or "it's going to" happen, as if they have a crystal ball. I am here to throw some water on this burning issue and explain why this is far from a foregone conclusion, and may not even be possible for anyone other than a nation-state.

At Black Hat USA 2026, I had the opportunity to ask various insurance carriers how many insurance claims are attributable to AI exploitation. I know Patrick Garrity has done some analysis on AI-driven KEVs as well, so I’ll defer to him there, but those are all known breaches, not specific to losses, which is an important sub-set of breaches. This is what the carriers told me when I asked them how many losses were AI-driven that had lead to a claim:

"Zero."

"Zero, literally none."

"We haven't seen any so far."

"We have seen zero as well."

“Zero and you won’t see much for the next two years.”

"I don't know of any claim to date that is specifically attributed to AI."

"None, unless you count phishing." I re-explained that I was asking about CVEs, to which he said, "None in that case. The only losses we saw attributed to AI were due to scraping, and they were claims for suits against the big frontier labs."

“If we're talking about AI-driven cyber attacks, we have not seen any attacks that resemble the OpenAI/HuggingFace attack. We suspect that attacks are using AI in discrete areas like crafting phishing emails. We know that threat actors are using deepfakes to clone executives' voices to attempt to convince employees to send money. Copyright claims that involve frontier labs (and even non-frontier labs that are nonetheless building LLMs) -- concur with your other source that these are the largest sources of losses that we are currently seeing. Seems like all that could change on a time with a large-scale AI-driven supply chain attack...we'll see!”

Together, these eight direct quotes represent approximately 30% of the global cyber insurance market. Fascinated, I was eager to share this information with others around the conference to get their take.

These were the objections I encountered.

The first objection I heard from people in Infosec was, "That is ONLY 30%; they may have different customers than the other 70% of the market", and "That remaining 70% may be where the losses are." Sure, while that is possible, it's important to understand that these eight very large cyber insurance companies collectively represent the low, medium, as well as the high end of the market with hundreds of thousands of policies, many of whom are based in different geographies, and represent different industries. Why attack some companies and not others based on the insurance carrier? This seems implausible to me.

However, if the event didn't appear in the loss data at all, because there were no losses, just breaches that didn't evolve into a claim, sure, that is plausible. That might very well be the case for nation states who aren't there to snatch purses.

The second objection I heard was, it's too early. Okay, I could buy this one a bit more, because we have only had about half a year with access to Mythos and claims do take time to fully realize. However, we have had LLMs that were capable of finding vulnerabilities since at least 2022, so that still seems extremely odd to me. I myself have found many exploits using LLMs, so I can confirm this is real and does happen. The CVE count has also shot up, which seems to indicate CVEs are being found by AI, so why aren't the adversaries using this embarrassment of riches?

My favorite objection was "their data must be wrong." It is not. Insurance companies are very good at collecting data most relevant to their margins. However, there are holes in the data. For instance, people being attacked may not have cyber insurance, or there wasn’t enough loss (or any at all) to warrant filing a claim, etc. A related objection is that many companies do not let the insurers know when they are breached, because they do not want any disclosure at all, and/or they don't want their premiums to go up.

One insurer I know on the smaller end of the market paid out over $600 million in one year alone. So companies do not want to disclose the breach in the case of AI, but they do in the case of non-AI exploitation? That seems extremely unlikely. Also, as an aside, it is illegal to withhold material losses if a company does business in a state that has disclosure mandates, so even more reason I think that this, while plausible, does not actually make sense based on the volume of claims being produced.

However, after talking with one DFIR expert, he confirmed the tooling necessary to detect if something was leveraged by an AI may not be up to par, and that is a fair potential hole in the data. I'll buy that one.

I think there are other, more likely reasons this may be zero and I have not heard them discussed elsewhere. The situation reminds me a lot of Fermi’s paradox, which states that if the conditions for life are so prevalent in the universe, then we should see it everywhere, and yet we do not. Similarly, the conditions for exploitation are so prevalent; we should see AI exploitation everywhere, and yet we do not. In Fermi's paradox, it is explained by a series of great and minor filters. An example of a great filter would be that the universe is too young to support life (and we are among the first), and a minor one might be that the composition of the Earth might be somewhat unique, diminishing the likelihood of life outside our own world. Likewise, I think AI exploitation has a series of filters limiting its use:

It's too early

The most common theory I hear for why we aren't seeing a lot of exploitation is that it's simply just too early. I believe this may be the truth, that we just haven't seen exploitation of AI-enabled CVEs or AI-malware appearing in the data… yet.

Once upon a time, attackers did not have access to web application proxies, and then suddenly, when they were finally made readily available, attackers started using them. I don't think this is at all shocking that, given a new tool, attackers will leverage them where it makes sense to do so after they become available. While tooling doesn't make exploitation fundamentally different and it changes nothing under the hood of the underlying ecosystem, it does measurably ease exploitation for those who didn't know how to code. Likewise, AI changes very little about the fundamental state of the environment it must attack, other than it makes it easier for non-technical people to attack targets; prompt injection and vibe-coded monstrosities filled with new vulnerabilities notwithstanding. So, I do expect to see more of this in the future; I just don't know that it matters much, if at all.

That said, we have had wide LLM availability since ChatGPT launched in 2022. There are millions of models on Huggingface, many of which are uncensored, and there are numerous tools to steer and manage open-weight models.

I asked a group of engineers at Black Hat what conditions must exist that don't currently exist that would enable AI-powered exploitation, and none of them could come up with any theories, other than it was just too early. That is a self-referencing argument.

It's too early, and the reason why it's too early is "because it's too early," is not a strong theory.

Training on the losers

Financial adversaries who work from open models are trained off of public data, and that public data is taken from bug reports, penetration testing blog posts, news groups and chat boards about infosec and write-ups from DFIR companies. None of these outputs are representative of what the current state of the art is, only whatever is already widely known.

Think of this cohort as "the losers." There is no way for the financial adversary to train off "the winners" because the winners do not get caught. To monetize an attack (such as ransomware), financial adversaries must announce their presence as the last step otherwise, no one is paying the ransom. That said, nation states do have access to the winners because they are writing their own payloads, and they understand what it takes to be stealthy, and that corpus of unique knowledge is used to train their models. For the financial adversaries, this is not an available option.

Where this shows up primarily is in a lack of stealth in the LLM and a lack of knowing how to bypass the most current state of the art. We have seen vibe-coded malware, to be clear, and it was awful - easily detected because it used ancient technology that has been detected for over a decade. We've also seen laughably bad malware that uses emojis. This is nowhere near a slam dunk at the moment.

I spoke with the CISO for a foreign nation working for their Department of Defense, and he confirmed that he does have access to the winners and he could theoretically train off of what they know that others do not. So, his team would not appear in claims data for that reason, and also because other governments aren’t filing claims when they catch one another, unlike financial adversaries who do end up in claims. We are seeing APT actors using LLMs, so this isn’t a fantasy. For instance, a Chinese APT appeared to use Claude, but they were both caught and produced no discernible losses and therefore no claims. Therefore bad, but far less bad than if they hadn’t been caught or had caused financial damage to the company that they compromised. That’s not a good outcome for the adversary.

Non-Determinism

One of the things that makes LLMs so powerful is the very thing that makes them almost a total non-starter. The last thing you want in your AI-enabled malware campaign is to have no idea if it will work or not, even in the presence of an easy vulnerability.

You don't know if it will attempt to attack you instead of the target. You don't know if it will attempt to turn you in. You don't know if it will delete itself, or the data you are after. You don't know if it will send the credentials off to competitors instead of sending it to you. You really have no idea at all what will happen. So, while powerful, it's also extremely risky and likely costly. Where a human may know exactly what to do, it may not, leaving a lot of money on the table for the would-be financial adversary. It does scale, but it scales in a way that may be the antithesis of what an actual adversary may need.

AI is difficult to steer without harnesses, and that requires a lot more sophistication and understanding of what needs to happen. That means skill... skills that you don't get unless you have already been in the trenches. The underlying theory that anyone will be made into a super-hacker goes out the window without some expert having written the harness. While an onslaught of hacking tooling may and even likely will happen, and may be released for wide use, it ostensibly doesn't exist today in the public domain, or if it did, it would presumably be in use. Because if it were useful, it would be used, right? At least that would be true if this were the only issue limiting adversaries.

To make my point in another way, OpenAI, Anthropic, Meta, and now even the open-weight models like Kimi are starting to confirm that they weren’t able to contain their models. If the companies most adept at building LLMs can’t stop models from doing unintended things, this is proving to be a dangerous solution to a problem that threat actors have already been solving using deterministic code. LLMs are dangerous because their out of bounds activity is likely to get the operators caught and/or the mission blown.

A good example of why this is an issue was shown in this thread by Gambit Security showing that while an attack using Claude was initially successful, it ended up denying service to a firewall and ultimately locked the adversary out. While the adversary presumably would have made money via ransomware had they not done that, they didn’t monetize the successful compromise by virtue of mistakes that the LLM made. That is a costly outcome. Said another way: the use of an LLM was the very reason a financial adversary failed to monetize a successful attack.

Or how about this example where a Chinese group tried to use open source models and had almost no success until they ended up switching to manual when their AI frameworks proved not terribly useful. It did save them hours of work in targeting, but who knows what it missed that a manual-only assessment would have found, or fully deterministic code. Ultimately, while it was technically a successful campaign, AI just wasn’t doing the job they expected it to do.

As one of the cyber insurance heads of underwriting told me jokingly, “Just tell Claude ‘don't get caught, monetize, make no mistakes’?” Right. Exactly. 🤣

Meme contrasting offensive AI in a lab, a menacing red ball, with offensive AI in the wild, a crudely drawn goofy red ball

We do see AI enabled scripts being used by Iran with regard to attacking PLCs in both the United States and the UK, but again, no losses. It’s not that losses cannot happen, it’s that nation states don’t have a financial motive. If you don’t have a financial motive, you can take your time, you can be much more calculating, you can create your own local models, harnesses, etc… trained on techniques that work.

Will nation states use AI? Yes, absolutely, and they already have. That’s not at all my point. The point is that it does not scale the way financial adversaries need.

Local model availability

Attackers cannot have access to Mythos for the time being, unless they break in and hack accounts of companies that are partners in Project Glasswing. But people will often hand-wave around this and then point to local models and there being no need to use Mythos specifically anymore. They will reference Kimi, Qwen, and other Chinese LLMs and "abliterated" models from Meta as being equally sophisticated and equally capable at finding security flaws as Mythos.

We have had a few months of these models being in the wild now. If their quality were approximately equal to Mythos, and if Mythos were as dangerous as it is made out to be, then there is no barrier to use. If there is no barrier to use, we should be seeing it used effectively. And we should start seeing claims by now. So why aren't we? Back to the previous theory that this is just too early?

Local model size

There are two places the adversary lands when they hack into a company. They either land on a perimeter device, like a VPN or Firewall, or they land on a desktop in the case of drive-by downloads or malicious attachments. In all of those cases, these are not typically machines that have a lot of RAM or processing power or disc. Typically, VPNs and firewalls are far less capable of generalized compute than an average business laptop, by a good long margin actually. They have almost no RAM, almost no disk space, and often aren't even running operating systems that can support generalized out of the box LLM platforms like Ollama and vLLM, etc.

Even if edge devices had enough RAM, and to be clear they do not, they also don't have gigs of unused disc available. It's easy to understand why: unused disc is a cost that hardware manufacturers don't want to spend, so they tend to optimize for very simplistic, hyper-performant hardware that are mostly driven by custom chips to eek out higher throughput. Edge devices are not optimized for generalized compute and do not have GPUs, for that matter. You need a lot of RAM or disc for an LLM. A small-ish general-use LLM is going to need about 16 gigs of RAM and around 8 gigs of disc. That's a complete non-starter on most edge devices. It literally cannot fit on most commercial devices.

Yes, you can fit much smaller models on even something as underwhelming as a Raspberry Pi, but these are very small quantized models that are pretty dumb. Keep in mind, this LLM is going to have to process gigantic amounts of hacked data too, which means it needs huge context windows. A Raspberry Pi or an edge device simply isn't going to cut it.

Desktops and laptops are a different matter. General purpose off-the-shelf compute can and does comfortably accommodate local models, and it is possible to install the software necessary to run them, but they are definitely NOT capable of running the largest and most sophisticated models that people say are analogous to Mythos. For that, you are going to need an absolute minimum of 64 GB of RAM and hundreds of gigs of available memory, and GPUs to accelerate the throughput enough to get even a few tokens per second. So yes, you can run a small local model, but you're not getting any kind of meaningful performance out of it, and it will lack the reasoning of the larger 1 trillion parameter models that have the huge context windows that people are raving about.

Said another way, this is just not possible. Not unless something fundamental changes about cost/need in RAM and disc to produce any given unit of intelligence via an LLM, because right now it's outside of the realm of the average business laptop or edge device, by a lot.

So that leaves external API options via Openrouter, et al, as the only current option. Doing so allows the adversary to use much larger models. Or there must be a new advancement that has yet to occur. To me, it seems clear that the truly local model is a non-starter for the financial adversary at the moment.

LLMs are noisy

LLMs are capable of a great number of generalized abstract operations, and that's great for the adversary in a lab or an academic/researcher. But it is far from optimal when you are trying to be stealthy. The very last thing an adversary wants is to start blasting out packets once they get into a network to do a massive port scan against all of the internal address space, or writing huge files to keep state, or a host of other bad ideas.

Yet, that is exactly what local models do, and how they keep getting caught. For instance, all of the frontier model companies have had their LLMs go out of bounds from what they were told to do, leaving TTPs all over the place, etc. They are so task oriented, that stealth goes out the window. However, stealth is the name of the game until the data is exfiltrated and/or the ransomware is in place.

Therefore, LLMs are quite easy to detect through MDR, threat hunting, deception tech, honeypots/honeytokens, etc. To be fair, these technologies are not as ubiquitous as they should be, but in larger orgs they are quite common. We have seen a number of AI-enabled attacks that did not lead to losses/claims, but all of them were caught. Getting caught is not ideal for the adversary, and is an evolutionary dead end.

Government intervention

At Black Hat, there was a private talk that more or less refuted the idea that models of the future out of China will be increasingly capable. In fact, the opposite may be true. I have no proof one way or another to be clear, so I will have to take his word for it, but his conjecture was that the latest models out of the Chinese labs have all stopped shipping with cybersecurity model cards, and are not attempting to win any AI hacking challenges, like they once did.

This could be just because they don't want to compete in this area, but a far more likely answer, which would explain the coordination across Chinese model providers and the conjecture of the talk was that the CCP has told the Chinese labs to stop shipping models with strong cyber capabilities outside of the country. If that is the case, we may have topped out on what the public Chinese LLMs will be able to do, and therefore what financial adversaries will be able to do, unless financial adversaries start building their own models.

To build their own models, we are talking about tens or even hundreds of millions of dollars of investment and months or years of work, and the tools and sustained talent to make it work, and of course the training data. Almost no financial adversary teams are going to do this, if any. It's not something they can do with stolen credit cards. No data center is giving someone tens or hundreds of millions of dollars of compute without knowing who they are, and that's not something most adversary teams are going to be comfortable with. Nor should they feel comfortable with such an idea; that's incredibly dangerous and likely to get them caught.

SAST vs DAST

There is a massive delta in what LLMs are good at versus bad at. LLMs are great at processing code. They are not at all good at blindly hacking something they have never encountered before or do not have source code for. For those who don't know these acronyms, SAST is static application security testing vs DAST, which is dynamic application security testing. Static means the code is at rest and not running; dynamic means it is. LLMs are fantastic at SAST, which means if you hand them an open source library, they can find vulnerabilities in that code, and they're quite good at it. But adversaries aren't attacking open source libraries, according to the loss data. At least not statistically meaningfully. No, they are attacking closed-source perimeter devices and closed-source operating systems, which require DAST.

An adversary has a glut of CVEs already. There are more than 370 thousand CVEs from which to choose from, and around a tenth of those have the proof of concept code available. So, it's not like adversaries don't have plenty of CVEs from which to choose; they do, and the code to use them. So why don't they?

The presence of a vulnerability does not mean it is in an application that is common, or in a configuration that is common, or that it gives the attacker the access they require to install the ransomware or steal the credit cards out of the database. Almost none of the CVEs fit the very specific needs of the adversary. And what they need is typically access to perimeter devices or a closed-source operating system, which is their foothold into the customer network, the very thing that LLMs aren't ideal at testing.

LLMs can produce CVEs extremely easily, but they are not great at producing exploit code that is useful to the adversary, at least not so far. Will that change at some point, yeah, maybe, especially as they get integrated with fuzzers and debuggers, et al. As source code for the otherwise closed source hardware and operating systems leak out to the Internet, this could evolve into an easier attack surface. But we apparently are not there yet, because there are still no losses associated with any vulnerabilities where the researchers who published said CVE said they were produced by an LLM. Not yet anyway! This will change, but because AI is taking over as the means of programming, not via some underlying landscape change.

But LLMs are great at finding slop - code that doesn't work at all. They're also great at finding low-severity issues that aren't useful to the adversary. Said another way, LLMs are great at wasting the adversary's time. I could see this improving, and likely will, but for now and until these issues are solved, the adversaries who invest in LLMs will likely see a decline in ROI, not an increase.

Conclusion

Will we see claims in the future for AI-enabled CVEs? Probably yes, is my guess. Will it be an absolute vulnpocalypse? No, likely not. Will nation states leverage AI successfully? Likely yes. These are not the same types of actors; their training data is different, their access to compute is different, their access to better models is not limited, etc. We need to keep in mind these differences.

My prediction is that we will see losses coming in the form of claims due to AI, but not for the reason most people may be thinking. Losses accrue, in some cases, when nation states are sloppy and get caught, which in turn forces disclosure due to, for instance, the Californian disclosure laws, leading to regulatory fines, and lawsuits from the victim’s customers… which ultimately converts into a cyber insurance claim. The nation state may not have been there to steal anyone’s money originally, but yet it still ends up in an insurance claim. AI is more sloppy and more likely to get nation states caught. Therefore, I predict that we are more likely to see insurance claims from nation states who use AI slop and are therefore more likely to get caught. AI is non-deterministic and therefore prone to exposing the operations of nation state hackers.

To be clear, we ARE seeing these attacks, but they aren’t leading to successes at scale. They are fast (like this one that was under 72 hours or this one that pivoted post breach within just 27 seconds), yes, but they are also very noisy and imprecise. That noise is the primary issue with doing this, not as a one-off, but doing it at scale, and doing it as a business. Research projects and one-off experiments do not represent what is practical.

There are some real-world issues that simply make it far too difficult today for the financial adversary. The defenders have time and the advantage of not having to be stealthy. The defenders are allowed to use whatever models they choose without having to worry about hardware limits on an edge device. The defenders do not have to live off the land, and they can feel free to work with large providers. The adversaries are far more constrained.

I do believe LLMs provide a very helpful role in other areas to the adversary, such as phishing/vishing, or processing the data once it has been exfiltrated, and perhaps vibe-coding back-ends of their criminal enterprises to manage campaigns, etc. However, I see no natural advantage in using it in the actual malware/attack itself that isn’t eclipsed by the downsides.

Due to these constraints, the popular theory of throngs of know-nothings becoming super-hackers due to open models appears to be a fantasy for the time being. Not that they won't try... It is that the use of LLMs in attacks is bounded by real-world issues that make such a proposition a bad idea.

Therefore, if anything, the data seems to indicate that we should encourage adversaries to use it, because it is so unlikely to work out for them financially. It's a costly decision to switch from something that has proven to work well, which has been to attack using the handful of CVEs that give them the beach-head that they need to pivot within the compromised environment.

Meanwhile, the totality of CVE-initiated losses we do know about, which are related to vulnerabilities in edge devices and operating systems, go largely unpatched in customer environments. I think we have got to get some priorities straight and focus on where 100% of the losses are instead of the zero percent of AI-enabled attacks that have caused losses. Unless your main threat actor is the government threat actor, it just doesn’t make sense, because it is either not in use or being poorly used by the financial adversary.

Said another way, Infosec spend should be focused on the probable, not the possible. Infosec should stop guessing.

← All posts