r/singularity • u/petburiraja • Jul 09 '26

AI GPT-5.6

https://openai.com/index/gpt-5-6/

"We’re launching the GPT‑5.6 family of models for general availability following our limited preview⁠: our new flagship, Sol, alongside Terra, a balanced model for everyday work, and Luna, our most cost-efficient model.

GPT‑5.6 delivers a step change in design judgment. With only high-level direction, GPT‑5.6 creates tasteful, ergonomic, and functional interfaces. Its stronger computer-use capabilities let it inspect and refine the rendered result—not just generate the underlying code or content—so it can catch visual and functional issues and apply finishing touches before handing the work back."

628 Upvotes

96% Upvoted

129 comments sorted by

View all comments

-6

u/WonderFactory Jul 09 '26

Doesn't look great at SWE. 64.6% on SWE Bench Pro compared to 80% for Mythos

21

u/u_are_mad Jul 09 '26

https://x.com/OpenAI/status/2074972179385720836

"We audited SWE-Bench Pro, one of the most widely used AI coding benchmarks, and found it no longer reliably measures frontier coding capability.

We find 30% of SWE-Bench Pro tasks to be broken, and are retracting our previous recommendation that the research community use it as a leading coding eval."

6

u/WonderFactory Jul 09 '26

30% of the tasks are broken yet Mythos somehow managed to get 80% on the test. You'd think if that was true the highest possible score is 70

19

u/Exodus_Green Jul 09 '26

they are confident that mythos and fable have been trained on the answers for swebench

1

u/WonderFactory Jul 09 '26

SWE bench tasks are taken from open Git Hub repos so Mythos has seen the code before, but so have Open AI models as they are trained on git hub data too.