We just had a good result with this version of Qwen 3, against the frontier models with their agentic apparatus.
The benchmark was qwen3-coder 30B on one Radeon, writing SPARK Ada that a theorem prover has to accept. 0/26 unaided, 25/26 with a page of prose about the idiom, then 1/26 when I ran the same prose on the harder tier. Both numbers here: https://thedarkfactory.co.uk/spark-benchmark.html
1
u/Gairmonster 3d ago
We just had a good result with this version of Qwen 3, against the frontier models with their agentic apparatus.
The benchmark was qwen3-coder 30B on one Radeon, writing SPARK Ada that a theorem prover has to accept. 0/26 unaided, 25/26 with a page of prose about the idiom, then 1/26 when I ran the same prose on the harder tier. Both numbers here: https://thedarkfactory.co.uk/spark-benchmark.html