Rob Joyce put out an interesting tweet the other day, and one line that struck me, “IR firms are saturated & turning away work.” I have no reason to doubt what he’s hearing, but it made me ask the next question. Why is that happening? Some might give a throwaway guess that “it’s AI” but I wanted to get to the bottom of this.
A few things to note, there may be more breaches occurring or more breaches being discovered/reported (the Breachpocalypse), but there is no meaningful increase in insurance claims thus far. I double-checked this detail with a provider the other day, and he confirmed that, yes, digital forensics and incident response (DFIR) is getting spun up more and there are recovery costs but for whatever reason there aren’t significant losses when the incidents occur.
Some of these reports might be because we have better tooling now. Some of it might be because we have higher reporting standards and laws. It’s hard to say because of confounding variables there. But losses are measurable, and according to the cyber insurance industry, they have not seen a marked increase in claims related to AI unless you’re counting business email compromise via phishing, though it is worth noting that there may be some issues with DFIR being able to mark some things as AI-sourced. There are other losses that are mounting, like privacy lawsuits when people don’t properly implement cookie privacy code on their website, but that’s not AI-related.
So what is happening?
I reached out to a friend who just quit his DFIR job and asked him some questions. He felt that Rob Joyce’s comment was true. Teams are overwhelmed by work, and the industry chats he is in are filled with people experiencing burnout. I asked the next obvious question, “Okay, and is it related to AI?”
“Yes,” he said. Then he went on to tell me all the problems AI has caused internally. He started by telling me about AI slop garbage reporting created by junior team members with hallucinated results that only the most senior people are qualified to refute. Keep in mind there are chain-of-custody issues, and they can wind up in court, so these details are critical to get correct. Imagine showing up to court and showing off vibe-coded reports full of errors and how the judge would feel about that. But the junior people are simply handing the totality of this sloppy reporting workload over to the most senior team members without curating it themselves.
Then we discussed sloppy AI vibe coding that has caused investigative code to disappear mid-investigation. It turns out that the people vibe-coding changes never use the features the senior people need and break critical forensic tooling without knowing why that matters.
Or even worse, investigations have themselves turned into cyber events because the AI agents these DFIR teams used went off the rails and given even slightly more access than they should have, caused them to break things, delete critical log files, create fake data, or all kinds of other horrible things. All of which are difficult or impossible to unwind, not to mention a massive time waster, and demoralizing.
I was so confused. This wasn’t a conversation about attackers using AI like I had expected it to be, this had turned into a conversation about the complete failure of using AI within the DFIR team itself.
I clarified, “Are adversaries using AI?” He nodded and was pretty confident they were, and that may be leading to more breaches, but it’s not necessarily leading to meaningful losses. Okay, great, now we’re getting somewhere, and that is confirmed by the insurance companies. We had a long discussion about the utility of AI, and he seemed very sure AI’s best place was as an orchestration layer, and not as the tooling itself. Said another way, AI is good at running tools, not being the tool. That jibes with what I have heard from various nation-state actors. Nothing new there from my perspective.
But he said that where they were once tracking dozens or hundreds of hacking groups, now there are thousands. What was once a fairly easy forensics exercise has become much more difficult, not because the adversaries are successful, but because they are spamming the logs with what is by all estimation malicious activity, but malicious activity that never had any chance of working. Nevertheless, it all must be looked at to find the needle in the growing haystack of forensic logs filled with malicious requests. AI has completely lowered the bar for attackers to at least attempt compromise, or even get a foothold, but beyond that it has not been proven to work at scale.
What happens after compromise is far less clear, because the AI-enabled adversaries don’t seem very adept at that part. Or maybe they are adept and simply haven’t gotten around to monetizing their wild successes, which seems less likely to me. A cyber insurer I spoke to subsequently believes that the reason even having initial access doesn’t make them successful in installing ransomware or pivoting to critical systems is because the existing defenses are still holding the post-breach line in the sand.
The DFIR expert I spoke to, and as well as two other former NSA TAO employees all believe that two major groups are benefiting from AI. The first is the nation-states, who can go even more under the wire than they used to. That makes sense because they can hide amongst the noise. They can even increase the volume of noise as well, to further make it next to impossible to find them amongst all of these false positive alerts.
The second group is the initial access brokers who are able to compromise more, and are now able to go down market, allowing themselves to hit smaller targets that wouldn’t have previously been economically viable to hit. So, the financial adversary utilizing agents is not an increased threat to the largest companies out there as much as the smaller ones - the ones below the security poverty line as Wendy Nather put it. But either way, it’s still not leading to more losses, for a variety of reasons, some of which are that these smaller orgs aren’t necessarily US-based, may not have insurance, may not report it, may not go above the deductible, and of course, it may simply be too early. We just don’t know, but the data doesn’t support this being a successful outcome for the adversary thus far.
Here is where things changed… I asked my DFIR friend how sure he was that these were real attacks, versus internal tools going rogue, or partners who accidentally let an AI tool off the leash. He thought about it for a minute and agreed that could be happening. Tal Be'ery similarly questioned if DFIR teams really are having to turn away work and at least one reply confirmed what my friend was telling me, that AI is “misbehaving and breaking things on their own” and it’s got a name now amongst the DFIR teams: “AI-induced destruction.”
As a point of reference, it has been his belief for a while now that if the case is ransomware, they can find the initial access vector in about 60% of cases, but where there is no ransomware, it falls to about 10% of the time. That means that 90% of the time they don’t even know how it began, because they lack the logs, the resources/time, the expertise, or whatever happened, to allow them to find out. My friend said AI use in his own organization had caused a number of incidents after all! During the investigations, if you give your investigative agent even 1% too much access, it goes off the rails and causes all kinds of downstream problems that are both difficult to detect and difficult to unwind. So how do we know that a lot of these cases aren’t a matter of self-inflicted wounds?
He gave me a story about a series of cryptocurrency investigations he had done, where 24 out of 47 incidents turned out to be technically “real” incidents but not attacker-related. Instead, the AI agent had accidentally caused an outage or transferred money out rather than an adversary triggering a loss. If that’s even vaguely close to what might be happening in other areas of the industry, it’s no surprise that we would see a massive spike in breaches that didn’t lead to a loss because there was no human involved and no intent, it was just an AI agent that went off the rails.
Similarly, Jeremiah had a conversation with an internal DFIR team where they found that roughly 80% of the internal security alerts were due to agents asking for macOS Keychain access, tripping the EDR alerts and causing an investigation. The internal teams have no choice but to investigate because they can’t know the difference between a legitimate AI agent and a malicious one, and in all cases it’s a waste of time. Talk about demoralizing!
Think about the DFIR industry as a glass of water with glass beads at the bottom. The glass height is a measure of its workforce capacity. The amount of water represents customer demand, the delta between the waterline and the top of the glass is the bench capacity, which is always kept as small as possible so that the DFIR companies aren’t wasting money. And the beads at the bottom of the glass are internal overhead, like meetings, coding/development, fixing unforced issues, etc.
The reason why my DFIR friend quit was because the beads at the bottom of the glass continued to grow. More work internally was caused by the AI tooling, and in his opinion, it was causing the work to suffer and he didn’t want his name on it, and the increased likelihood of catastrophic failures, so he left. More beads at the bottom of the glass created churn, and the most senior DFIR expert in the company left! When he did so, he reduced the height of the glass. That is a double whammy - more beads are added to the bottom of the glass, and the glass shrinks at the same time. The water was still there but needed somewhere to go, so it overflowed into another glass which in this analogy means sending the work to another DFIR company with spare capacity. If that additional workload can be absorbed, great. But if their glass is also getting filled with beads, because they too are using AI and it’s causing more churn… Well, Houston… we have a problem.
Also, if the water flowing in increases, there is no fast way to deal with the additional capacity needs because hiring and training people is slow. So if AI is creating a whole bunch of fake alerts from people who have accidentally hacked themselves, regardless of whether they reported it as a loss or not, it wasn’t a loss in the financial criminal sense of a loss. This would explain why the cyber insurance industry hasn’t seen it in their data. It would also explain the burnout because the bulk of the increase in new investigations doesn’t result in finding more bad-guys. Most DFIR people want to solve crime, not clean up after AI slop.
What has happened is internal processes grew more annoying to the most valuable DFIR employees - the most senior ones. That then heightened the need for more AI, which caused more problems and the cycle will only worsen as more DFIR experts leave. This has caused DFIR to quickly decline in utility and confirms what Rob Joyce has heard. There’s virtually no DFIR capacity left, and it’s dwindling. An overflow of capacity from even just a handful of teams can easily overwhelm the rest of the remaining DFIR teams, leading to industry-wide burnout, but never having made meaningful gains on identifying causes of attacker-originated loss.
Okay, all that said, I am NOT saying that this is true, I am saying it is one theory that happens to explain everything that we see externally rather well. That is not ideal though, because it also means it will be wildly easier for the adversaries because the best and brightest are egressing out of the DFIR industry due to burnout.
Let’s review. If true, this also means that AI has:
- Caused an explosion of new CVEs irrespective of their utility to adversaries
- Caused an incredible additional workload on the average DFIR team irrespective of actual adversarial activity
- Caused unforced errors that resulted in their own incidents
- Made it far more difficult to get triaging done because teams are overwhelmed with self-inflicted wounds with no easy way to tell which is real and which isn’t until deep in the investigation, if at all
I ran this theory by a different DFIR expert and he nodded, confirming at least part of this effect of junior people overloading senior people with AI documents that aren’t fact checked, which is tiring and often riddled with errors which causes burn-out amongst the most senior staff. The senior staff are the only ones qualified to deal with these reports and apparently the only ones reading them, since the junior people are relying on AI almost entirely. No wonder they’re getting burnt out and leaving, if that is the case, and if so no wonder DFIR capacity is waning, and no wonder they are being forced to turn away work.
And no wonder that DFIR may be on life support, with no hope in sight.
I’ve been trying to get feedback on this theory for a few weeks now, and the best feedback I got was from Eric Taylor of Barricade Cyber Solutions, a boutique DFIR firm. He said, “There are a number of smaller DFIR firms that have plenty of room to handle new cases, and these smaller firms most of the time prefer the SMB market as they are able to really focus on their needs.” If true, that would lead me to believe that perhaps the larger players are more focused on using AI, likely in a misguided attempt to reduce costs and increase bench capacity. These larger players may also be prone to sharing deals with one another because they know they typically can handle larger numbers of cases at a time. That may be causing problems at the top end of the market, while the bottom end is currently less affected or even completely unaffected.
Eric also added, “I am also aware there has been a lot of news about questionable actions at various DFIR firms. To get around this they have just rebranded themselves.” I have no idea what this implies for the global need for DFIR but it may be that we are seeing DFIR rebranded to AI SOC companies or something else, to live another day under a new brand, a new valuation and without the same regulatory mandates for chain of custody. But that’s entirely speculation.
I would love to hear your thoughts.