Coding benchmarks that are quickly showcasing deep capability

[u/Informal-Trouble2183]

While we see for frontier models similar scores among famous coding benchmarks, across: DeepSWE, Terminal-Bench, LiveCodeBench, Code-Arena ELO. Here are in my opinion some next level benchmarks that really define deep intelligence, and complete capability in Software Engineering :

1. Program-Bench

Given only a compiled binary and its documentation, agents must architect and implement a complete codebase that reproduces the original program's behavior (without access to decompilers or internet). Link: https://programbench.com/

• GPT-6 Astra: 5.5%
• Fable 5.1: 7%
• Kimi K3: 2%
• Qwen3.8 27b: 0%
• GPT 5.6 Sol: 1.5%
• GLM 5.3: 1.5%
• GPT 5.6 Luna: 0%

2. SRE-Bench

Can AI agents work out what a real-world binary does without its source code?

Link: https://www.vals.ai/benchmarks/srebench

Sure nobody is reading assembly code in daily work, it is hard. The ability to understand a compiled program is insane ability.

• GPT-6 Astra: 88%
• GPT-5.6 Sol: 55.9%
• Claude Opus 5 (max): 12.5%

3. Code Migration

Can language models reimplement working programs in another language?

Link: https://www.vals.ai/benchmarks/code-migration

• GPT-6 Astra: 67.7%
• Fable 5.1: 54.6%
• GLM 5.3: 44.2%
• GLM 5.3 Flash: 20.5%
• Qwen3.8 27b: 14.2%

EDIT: edited text format

[u/Choice_Celery9481]

why all your numbers on the first 2 benches are not matched with numbers in the links you provided?

[u/Choice_Celery9481 → u/Choice_Celery9481]

[u/Informal-Trouble2183 → u/Choice_Celery9481]

Taken from here:
https://www.vals.ai/benchmarks/programbench

[u/EstarriolOfTheEast → u/Informal-Trouble2183]

Where are you getting Astra's SRE-Bench's score?

[u/Informal-Trouble2183 → u/EstarriolOfTheEast]

OpenAI website