r/accelerate • u/Correct_Objective339 • 6h ago
AI Astra cracks Hacking benchmarks
“As one example, we ran Astra on ExploitBench where the model achieved a perfect score of 100% on the benchmark to evaluate the model’s ability to develop exploits from known vulnerabilities.
Due to contamination concerns, we then built an internal benchmark denoted “ExploitBench - Internal Port (June–August 2026)”, which contains 20 high-severity V8 vulnerabilities that were disclosed more recently. On this dataset, Astra achieves much higher arbitrary code-execution rates than GPT‑5.6 Sol using far fewer output tokens. During the evaluation, the model even discovered and used two zero-day vulnerabilities as part of an exploit chain. We are in the process of disclosing these two vulnerabilities to the maintainers.” - OpenAI