And are we stumbling into an uncanny valley?
In July I wrote about the lessons cyber can take from the SRE transition and promised a follow up on how AI accelerates the path from limited automation to full autonomous operations.
Here is that blog, with a perhaps surprising bottom line upfront: the crucial pieces are less the agents and more the architecture, including the data layer and the learning loop, and the changing roles and activities of the cyber SREs. Progressive organizations are reskilling their teams now and thinking from first principles about architectures for autonomous operations; those that focus solely on accelerating existing processes with AI risk stalling in an uncanny valley, where humans lose trust and belief in the system.
“Agentic”, “Autonomous”, what’s the difference?
In cyber marketing, agentic SOC and autonomous or machine speed cyber are often used as synonyms. They are fundamentally different as I see it.
I think we can agree that an Agentic SOC is a SOC in which agents do work. They triage, enrich, investigate, draft, and, typically with a person's approval, act. Autonomy, on the other hand, is a property of each agent, typically granted or withheld one action at a time. Most AI SOCs are Agentic SOCs, and they do improve the tab switching, “swivel chair” workflows they replace.
My experience in the adoption of autonomous cloud and devops tells me that an autonomous SOC is not simply a SOC where an agent acts autonomously sometimes. Rather, as we see it at DeepTempo, an autonomous SOC is a cyber security defense system in which the system as a whole meets a declared objective, and improves on its own, without a person needing to approve the vast majority of individual actions. We see autonomy as a property of the architecture. Agents inside it may be tightly constrained or largely free, and the license to act is defined, versioned, enforced, calibrated, and adjusted by the system rather than negotiated per incident.
By the way, I don’t deserve any credit for this delineation or definition. Google's SRE handbook codified much of what was in the water a decade ago, and draws a similar line. In the chapter on the evolution of automation, the authors call automation a force multiplier and then argue that a higher order design requiring neither manual operation nor automation, an autonomous system, beats both. The Agentic SOC is the automation stage. As we see it at DeepTempo, the autonomous SOC is simply the optimal autonomous system doing cyber defense.
Great section heading from the SRE handbook, worth a read, it is canonical:
Another pattern from the past that is fairly intuitive: autonomy should advance through tiers. Inform only, then human in the loop, then human above the loop, where people set mission, policy, constraints, and escalation rules and the agent operates inside them. There has been a bunch of text and tokens spilled on this point.
What follows are the parts of the journey to autonomy that are less obvious, drawn from years of watching StackStorm, which I founded and led, run in production at Netflix and elsewhere. We learned a lot from various DevOps teams over the years, including:
1. The approver should not be the safety mechanism.
At a 2015 meetup Netflix hosted to present Winston, their StackStorm based diagnostic and remediation platform, one of the first questions from the audience was "who watches the watcher?"
The answer was not "a person approves each remediation." Netflix could not have reached their scale that way. The answer was architectural: runbooks written as code, reviewed and versioned; remediation gated on verified safe conditions; authorization and audit built into the platform; and a culture that treated a misfire as a reason to improve the platform rather than to add an approver. If the work is done well upstream, then the operations code works and is trusted.
Across the broader SRE and devops and StackStorm community the pattern repeated. Teams that reached for auto remediation before their actions were idempotent and reversible got burned, and often retreated to approval gates for years. Teams that invested first in the platform, in typed actions, in packs isolated per team, in promotion through dev, test, and prod, kept expanding what ran unattended.
2. Visibility was more important to autonomy than approval gates.
At least in the cyber world, this seems to be non-consensus. ChatOps, which StackStorm helped popularize, put automation's every step into a shared channel where the whole team could watch it work. This turned out to be central to developing trust. Once people had watched a runbook succeed a few hundred times in public, the approval step began to look like theater, and teams removed it on their own. Transparency, not permission, is what converts a supervised action into an unattended one.
3. More change, more stability.
One of the most impactful findings from DORA’s research summarized in the important book Accelerate found that the teams changing production most often also broke it least, because small reversible changes are testable and cheap to undo, and the reflexes that catch failure stay warm.
The SRE handbook is clear about this: automation that runs infrequently, cluster failover for instance, is fragile, since the feedback loop is too long to trust. As far as I can tell, security operations follow the opposite instinct. Detection logic freezes, playbooks calcify, automation improvements wait for a quarterly review, and as a result, most SOC systems change rarely, are brittle, and break badly.
4. Declarations and contracts and loosely coupled architectures scale; people don’t.
Scale comes from defined interfaces, loose coupling, and declarative desired states. The defined interfaces are emerging in cyber, thanks in part to MCP and platforms like Cribl that unify data access and more. However I’m not sure we have a product mentality yet within cyber for subcomponents of the loosely coupled stack which is also important for this pattern to work.
That’s assuming the stack is loosely coupled. We often see systems where every step of the workflow needs to complete for the automation to perform; this, of course, is one of the ways that SOAR became brittle and overcoming this with intelligence is a major promise of AI powering cyber ops.
The declarative pattern is also deceptively simple. Just write down what the world should look like, let a controller reconcile reality toward it, and have people review changes to the declaration rather than to the reconciliation. Kubernetes is a case of this architecture. The controller loop is the autonomy. The reviewed manifest is the control. Actually using these manifests intelligently, for example running them through their own pipeline, is fundamental for this gitops related pattern to deliver.
The importance of new architectural and operational patterns is why I see many orgs getting stuck in an uncanny valley. A SOC that just tries to “pedal faster” will end up with a faster ticket queue, or patch loop (btw check out the patch loop our Sam Armstrong open sourced, learn more about it on SOCbench.org); the result can be human revulsion, or at least confusion, often manifested by especially non-technical executives feeling like AI hasn’t helped their cyber much.
We need to leap across this somewhat uncanny valley. Bespoke management of agents must fall away in favor of architectures that enable the safe and ever improving use of hundreds or thousands of agents. Help your team transition to curating the skills, the policies, the evaluations, and the license to act. We need to leverage our best architects and the vendors with the best operational experience at scaling autonomy to succeed (subtle, right?).
Skills as desired state
The first generation of AI SOC solutions were built in the time of chatbots and before last October’s release of skills by Anthropic.
Vigil was built in a time of smart reasoning models and skills. Given our background in helping to build gitops environments, building around skills always made sense.
We made Vigil “skills-native”. Vigil uses skills to track declared intent as markdown under change control, per environment. A skill declares what the agent is for, what evidence it must gather, what it may do, what it must never do, and when it must escalate. It is the SOC equivalent of a manifest. Agents reconcile toward it. When an analyst wants to change how phishing is handled, they open a pull request against the skill, not a ticket asking for another approver. And this primitive can be built upon. The same architecture, more abstraction, better declarations.
We also use skills for workflows, so essentially the combination of agents are also, underneath, controlled by skills.
We increasingly see the control of skills replacing the need for frequent approvals of actions. Review moves from "should this host be isolated" to "should hosts matching these conditions be isolated by this agent, in this environment, with this rollback." That is a higher-leverage question, one a person can settle in minutes, replacing the need for one recurring many times a night. Mayank's write-up of Aron Eidelman's talk at the recent San Francisco AI Cyber Alliance meetup describes Google’s similar approach in a neighboring domain: skills for threat modeling and secure coding that port across coding agents (Agent skills and Ice Houses).
Skills outlast the models and harnesses that run them. Much of the software will prove disposable; the choice of model can change as well; and the system can self-optimize over time; so change is a constant. The declared intent becomes amongst the least changing aspect of the autonomous SOC.
License to act is per skill, not per SOC
What decides a skill's tier and degree of autonomy? We look at blast radius, reversibility, evidence and other inputs that the user can configure.
Evidence is where a Detection Level Objective (DLO) earns its keep. A skill promoted from shadow mode to bounded autonomy should carry a measured precision, a measured false positive rate, and a canary history, the way a service carries an SLO before it takes production traffic. Demotion is one incident away. Promotion requires a body of evidence.
Paved paths
Another pattern that likely will emerge in the building and operation of an autonomous SOC is the idea of paved paths. StackStorm as an auto remediation solution was made available to other groups within Netflix, for example. They didn’t have to use it. They could cut the corners, forge their own path. They were still accountable for the performance of their subsystems, though, so many decided to focus on their core capabilities as opposed to attempting to build their own resilience system.
One way a paved path pattern is emerging in the autonomous SOC is in the way that agents earn autonomy. On the paved path a skill has evaluations, a canary environment, rollback, structured audit, and telemetry the LogLM can consume. Off the paved path an agent can still run, but it cannot accumulate the evidence to be promoted. Such an agent begins as a canary, running in ghost mode, and then over time may be able to be more fully trusted.
The data layer carries the journey
The architecture, and above all the data layer, for an autonomous SOC needs to be designed for the needs expressed above from the start.
Mayank's account of training a model on data the adversary refuses to give you makes the point for detection: treat data preparation as the product, run evaluations continuously, and promote nothing on intuition. The same discipline governs the SOC as a whole. Dispositions, outcomes, canary results, and red team runs have to land in a data layer the defending organization owns, in a form the evaluation factory can consume, or the journey stalls at the approval gate forever.
Trust is the second requirement. You cannot grant bounded autonomy to a system you cannot inspect, which is why the architectures that carry organizations across this journey tend to be open source and understandable down to the interfaces the agents act through. Vigil is Apache 2.0 for that reason, and its skills are readable Markdown for that reason.
Learning is the third primary requirement, and in the world of AI it must be a first-class citizen of the architecture rather than a vendor's background process. This is challenging for most SOCs since, obviously, they cannot be expected to be reinforcement learning experts. While we see national SOCs and major regulated entities aware of the critical importance of owning their learning loop, many organizations are either unconcerned or uninformed about the learning loop.
All of the above architectural elements, including the learning-enabling data layer, are owned by the Security SRE. In an Agentic SOC a person owns actions: approve, reject, escalate. In the Autonomous SOC the team owns outcomes, which means owning the learning loop that improves them: which dispositions feed back, which evaluations gate promotion, which skills get revised after an incident, how the detection layer itself is tuned on the organization's own telemetry. Owning the loop is the mechanism by which owning the outcome is possible.
The security SRE, expanded
That shift towards owning the loop can be empowering for the security teams. An analyst who approves actions has a remit the size of a queue. A security SRE who owns the learning loop has a remit the size of the system: detection quality, skill design, evaluation, the paved path, the DLO. The original SRE transition lifted the people who made it, in impact and in compensation, because operating a system and improving it became the same job. The same lift is available here, to anyone willing to think across the whole architecture rather than the next alert.
Destination
Autonomy is earned, and the entity earning it is the architecture, not the agent. Agents will turn over with each model generation. What persists is the set of declared intents, the interfaces they act through, the evidence that licenses them, the data layer that holds that evidence, and the paved path that makes the safe route the easy one. That is what allowed distributed systems to run the cloud with a fraction of the operators the old world required, and it is what will let a SOC run at machine speed with people doing the work only people can do.
If you want to see the shape of it, Vigil is Apache 2.0 and runs in your environment with your models. Skills are Markdown. Pull requests welcome.
I’d welcome any and all feedback on this blog. This was hand crafted and I cannot blame AI for any confusing elements. Please let me know if you enjoyed it and whether you learned anything from it.
