The Golden Age of Threat Hunting

Hunting for everyone

We may well be in the golden age of threat hunting.

The headlines this year have gone to offense. Kevin Mandia's Armadin launched in March with an AI attacker swarm and has since raised $445 million at a valuation north of $2.5 billion. Good for them; continuous red teaming is worth doing. Meanwhile the AI SOC crowd has spent most of its energy automating alert investigation.

Hunting has long been something of a luxury good in security operations: everyone admires it, but only the largest banks and government agencies can afford to keep hunters on staff. Good hunters are expensive, scarce, and get pulled back onto the alert queue the moment things get busy. Agents remove that constraint, and hunting turns out to be amongst the safest places in the SOC to let them run.

In this post I revisit a framework that helped first give hunting a shape, Sqrrl's Hunting Loop, and share how it helped inspire the design of Vigil, the increasingly popular open source AI SOC.

Along the way I compare hunting with its better-funded cousins, investigation and red teaming, and show how hunting is emerging as a safe workflow for SOCs moving from human-in-the-loop to human-on-the-loop automation, that is, from agentic toward autonomous operations.

A loop worth stealing

Around 2015, back when I was still building StackStorm, we heard about Sqrrl (later acquired by Amazon) and its Hunting Loop. It was elegant in its simplicity:

1. Create a hypothesis about what an adversary might be doing in your environment.
2. Investigate it with whatever tools and techniques fit.
3. Uncover new patterns and adversary tradecraft.
4. Inform and enrich your automated analytics with what you found.

The fourth step may be the most important. It fits the SRE mentality of automating away toil, and anyone who has run operations will recognize the pattern: find the problem once, then automate it out of existence.

Sqrrl paired the loop with a Hunting Maturity Model running from HMM0, where you rely entirely on your alerts, to HMM4, where most successful hunts become automated detections and your hunters spend their time on new questions. Eleven years later, it seems that most organizations sit at HMM0 or HMM1.

Side note: the framework aged well enough that one of its authors, David Bianco, gave an RSAC 2024 talk discussing PEAK, its successor at Splunk. PEAK adds a hunt type close to my heart, model-assisted hunting, which is more or less what Vigil does when it hunts alongside our LogLM.

Why hunters end up answering alerts

Survey after survey says the same thing:

• In the SANS 2025 Threat Hunting Survey, 61% of respondents named a shortage of skilled staff as a primary barrier.
• Intel 471 points out that hunting is often last in line for budget, and cites a SANS finding that 75% of organizations hunt with people who have other day jobs.
• SC Media reported that only larger organizations can pay the roughly $250,000 a year an experienced hunter commands.

Shift left: investigate, attack, or hunt

Most SOC labor has always gone to investigation, and from what we can see, it still does. In one 2025 survey of 282 security leaders, the average organization fielded about 960 alerts a day, spent roughly 70 minutes investigating each, and left 40% uninvestigated. That arithmetic explains why the AI SOC category exists and why automating investigation seems like the first-order problem.

Once investigation is automated, the question becomes where your people, and your agents, go next. A logical answer is to the left, toward work that finds trouble before an alert does. Here are a few comparisons of investigation, red teaming, and threat hunting that can be useful in thinking through when and how to apply autonomous AI:

Red teaming and hunting are complements. A red team shows which doors an attacker could walk through; a hunt checks whether someone already walked through one. Both can surface the need for better detections, and a mature system ties them together: paths the red team finds become hypotheses worth hunting.

The fifth row is worth highlighting. A red team agent that errs can take down production, which is why red teaming runs under rules of engagement. A hunting agent that errs wastes a query and possibly increases the risk of a missed attack.

How Vigil runs the loop

The short version: Vigil runs the first three steps of the loop today, and most of the fourth. You can take a look yourself; it's all in the repository, Apache 2.0 licensed.

The Hunting Loop as Vigil runs it. Solid paths run in Vigil today; the dashed path, from proven hunts to new detections, is where we are heading. More on that below.

Step 1: Start with a claim worth testing

A Vigil hunt won't start without a hypothesis. Someone, or something, has to say what we're looking for. The hypothesis can come from an analyst, from a threat report you drop into /hunt (Vigil pulls out the report's own hypothesis, or writes a testable one), from fresh indicators in your threat feeds (proposed and held for a human to approve), or from a schedule.

A couple of details I'm especially fond of. Before a hunt launches, Vigil checks whether someone is already hunting those entities and techniques, or has already settled the question. And every hunt starts with a rival hypothesis built in: the boring, benign explanation. That's the one to beat.

Step 2: Investigate without fooling yourself

A Hunt Lead runs the show, and it never touches telemetry directly. Each turn it picks one move from a short list (investigate, pivot, deepen, abandon, validate, hand off to incident response, and a few others) and sends up to four specialists to work in parallel. One works the findings and raw telemetry, one studies the shape of the traffic (beacon intervals, jitter, bytes out against bytes in), and one checks threat intelligence. LogLM optionally plugs in through MCP, contributing its malicious events and similarity search, so a hunter can take one suspicious sequence and find every behavioral relative of it across the environment.

The part I'd point a skeptic to is how Vigil keeps a hunt honest. Human hunters fall prey to confirmation bias; agents essentially do too, only faster. To counter this, Vigil makes the Lead rule on every new piece of evidence against every live hypothesis, including the benign one, before it can ask for more. Each finding records who wrote each value it rests on: the sensor, the adversary, or a third party. Vigil won't let a hunt drop a lead on attacker-written evidence alone.

Step 3: Make it survive a skeptic

When the Lead thinks it has something, it doesn't get to declare victory. Vigil calls in an independent critic whose only job is to argue the strongest innocent explanation it can find: a backup window, a vulnerability scanner, a proxy rewriting addresses, plain old misconfiguration. A hypothesis is proven only if it survives that argument and is corroborated by more than one telemetry domain. Only proven hypotheses go to incident response. Your IR team will thank you.

Step 4: Remember what you learned

When a hunt wraps up, Vigil distills it into verdicts tagged with ATT&CK techniques, so the next hunt (or the next analyst) can ask what we already know about a host or a technique. It records what the hunt couldn't see. It also records hunts that found nothing, which is more valuable than it may sound: it keeps you from hunting the same ground twice, and it gives your auditors evidence of coverage. And you can replay every decision the Lead made, with exactly what it saw at the time.

From agentic to autonomous: why hunting goes first

We've written before about the gap between agentic and autonomous security operations. The short version: agentic systems do the work, autonomous systems own the outcome, and autonomic systems improve themselves. In Vigil, autonomy is earned skill by skill, on measured evidence, and only a human promotes an agent from one tier to the next: inform only, then human approves, then bounded autonomy.

We are seeing hunting emerge as a use case where promotion happens surprisingly often, for at least three reasons.

It's non-destructive. A hunt reads telemetry, findings, and intelligence, and it changes nothing in your environment. The worst a confused hunting agent can do is waste a query, so its license to act can be broad from the start.

It's naturally parallel. Every hypothesis, entity, and technique is its own hunt. Investigation is bounded by how many alerts arrive; hunting is bounded only by how many good questions you can ask. There's no reason to run four hunts when the question space supports four hundred, or four thousand.

It compounds. Before every hunt, Vigil checks whether the ground has already been covered. After every hunt, it records what was found, what wasn't, and what couldn't be seen. Run enough hunts and the swarm stops repeating itself: each new hunt starts from what the last thousand learned and aims at the gaps they left behind. The program gets cheaper and sharper the longer it runs.

Hunts still cost money. Every agent spends tokens, and every query lands on a SIEM somebody pays for. Vigil today is deliberately conservative here. By default Vigil assumes you would want to run four hunts at a time, each with up to four workers, under an hourly cost brake that keeps feed-driven hunts from crowding out critical detections. The coverage memory is what makes turning that dial up worthwhile, because Vigil's swarms know to focus on new ground.

Notice that this is the move from agentic to autonomous in practice. In an agentic SOC, a person asks for a hunt and approves its steps. In an autonomous one, people declare what the hunting program is for (which assets, which threats, what budget), and the system decides which hunts to run, runs them, and reports what it learned.

The last mile: from hunt to detection

Vigil doesn't yet turn a proven hunt into a detection automatically. We and the broader Vigil community are moving in this direction and are extremely excited about how LogLM can help with superior detection engineering. In some cases Vigil is already becoming a purple team tool, and we are excited about the potential of these experiments.

Returning the initiative to defenders

Sqrrl drew a great loop a decade ago. Since then, what has kept most SOCs stuck at the bottom of the maturity model for threat hunting has been the price and scarcity of the people needed.

Agents with a smart harness remove that constraint, and hunting may be the safest place in the SOC to let them loose. The agentic red teams will keep the headlines for a while; my bet is that hunting quietly becomes a standing practice at organizations that have never had a hunter on staff.

Am I right? I'd love to hear from hunters, especially the skeptics. Where would you trust an agent in the loop today, and where wouldn't you?

Take it for a spin

Vigil is open source under Apache 2.0. Clone the repository, point it at your telemetry, and run a hunt. And if you have ideas on how to extend Vigil, please come build it with us on GitHub or in our Discord.

Sources

• SANS 2025 Threat Hunting Survey (key findings summarized by ThreatDown)
• Splunk details Sqrrl 'screw-ups' that hampered threat hunting, TechTarget
• PEAK Threat Hunting Framework, Splunk
• The Threat Hunting Conundrum, Intel 471
• Why good threat hunters are hard to find, SC Media
• Threat hunters struggling despite increased security funding, SC Media
• The State of AI in the SOC 2025, The Hacker News (contributed by Prophet Security)
• Kevin Mandia's Armadin launches with $189.9 million in funding, SecurityWeek
• Armadin raises $255.5M at a $2.5B-plus valuation, Runtime Wire
• Vigil-SOC/vigil, GitHub