Three posts from our team on the most crucial part of building a detection model: the data.
Training a Model on Data Your Adversary Refuses to Give You
Mayank Kumar · August 6
Building the LogLM model was challenging; feeding it and evolving with better data over time may be even more challenging. Mayank draws on the data engine playbook from autonomous driving, Tesla's shadow-mode fleet and Waymo's closed-loop simulator, and walks through the five-stage flywheel behind LogLM: discover, process, simulate, evaluate, feed back.
The place the analogy breaks is perhaps the most interesting part. Tesla's data source is cooperative. Drivers want to drive well, so every mile generates an honest signal. Our subject actively works to not generate data, blends into benign traffic on purpose, and changes behavior the moment it is detected. Autonomous driving mines edge cases from a willing world. We manufacture ours from an unwilling one.
Read here
Failures or Failsafe? A Perspective from the Arena of AI Detection Evals
Jack Murphy · August 10
What happens when a strong model posts a terrible score?
On a customer eval spanning more than 150 malware variants, LogLM appeared to miss most of the malicious traffic. The handful it did catch showed a strange signature: single packets registering zero bytes. That thread unravelled the whole result. Packet header data had never made it into the logs, and roughly 62 percent of the malicious labels turned out to be wrong.
The same pattern emerged on public data. LogLM's embeddings surfaced simultaneous benign and malicious labels on the same attacker IP in NF-ToN-IoT, and an entire SlowLoris DDoS labeled benign in SIMARGL2021. Jack demonstrates that a model able to separate behaviors can double as a detector of bad data, and that detection failures can become useful signals.
Read here
Building the Evaluation Factory Behind LogLM: TempoRange
Christian Moran · August 6
TempoRange is a segmented enterprise (corporate AD, OT/ICS, IoT, management networks) running a continuous benign baseline alongside three attack engines: scripted kill chains across MITRE ATT&CK with exact labels, a detonation range running live malware under layered egress containment, and an ARTEMIS range built on Stanford's autonomous red team agent, dropped in with no script and left to improvise at machine speed.
The payoff Christian points at: when a new technique is published, it can be reproduced in the range and run against the model inside the same news cycle. This enables us to confirm to our clients whether their LogLM is indeed seeing the attack everyone is talking about. A useful read for anyone thinking through AI based ranges and data quality.
Read here
