THE ESSENTIALS IN 30 SECONDS

  • Vals AI's CyberOffense benchmark has AI agents attack two simulated enterprise networks with 21 and 31 objectives, under a 500-million-token or 48-hour limit.
  • OpenAI's GPT-6 Sol, available only through its restricted Daybreak cyber program, led; the downloadable DeepSeek V4.1 Flash was close behind and level on the first network.
  • DeepSeek runs cost under $3 against about $22 for Daybreak on that network, Vals says; the benchmark relies on only two ranges and scores vary between runs.

Openly downloadable Chinese AI models are getting close to the best restricted American systems at breaking into simulated corporate networks. WIRED drew attention to the trend this week, citing results from Vals AI, an evaluation company whose CyberOffense leaderboard was last updated on October 23.

What the benchmark measures

Vals built two simulated cyber ranges. "Nova" spans several connected Active Directory domains with 21 objectives; "Lighthouse" covers two separate corporate networks with 31. Each model drives the open-source Pi agent from an isolated Kali Linux machine that can only reach the range and the model provider, and works through a multi-stage intrusion: initial access, privilege escalation, password reuse, moving between machines and abusing trust between domains. A run stops after 500 million tokens or 48 hours, and an objective only counts when a hidden marker shows up in the logged command output. The authors say they reviewed every run and found no shortcuts.

The results

GPT-6 Sol, which OpenAI offers only through its restricted Daybreak cyber program, topped the table. Its best run completed 27 of 31 Lighthouse objectives in under three hours. DeepSeek V4.1 Flash, an open-weight model, came second, and on Nova it matched Daybreak on average (16.3 objectives against 15.7). Averaged across both ranges, Dealroom's summary of the coverage puts the two at about 73% and 81%. Kimi K3.1 Pro, Meta Wonder and Grok 4.7 followed; GLM 5.3 Flash completed five objectives on each range.

Cost is the striking gap. According to Vals, a DeepSeek run on the first network cost under $3, against about $22 for Daybreak. The company argues that cheap open models can keep working for a long time, so short tests understate them, and that once weights are public there are no provider safeguards to stop misuse.

Limits

The benchmark rests on two ranges, and scores varied between runs: Daybreak completed between 12 and 19 Nova objectives. Cost figures are published for only two models. Several models stalled before exhausting their budget, and no model finished the last Lighthouse objectives. Dealroom also notes that results in a simulated environment do not guarantee success against live, defended systems. Vals presents the findings as a window for defenders to test and fix their own networks, and expects attacks of this kind to become more common.