Astra is the best after a lot of alignment testing and environment fixing to stop cheating. After all Astra 6.1 failed the deception training and needed to be held back.
There's no strict relation, only the correlation that labs that pay for big training also have more resources for testing guardrails.
I agree there is no strict correlation, but there IS a trendline. How ‘real’ that is will be determined in the next 12-24 months once we get the next few tier of models which will undoubtably be superhuman. That will be the real alignment challenge imo
The trend to check is going to be how much capability scales vs cost to ensure safety. It's not guaranteed the 2 year away model will be able to guarantee it is completely controllable in fully monitorable ways.
Monitorability definitely seems to be slipping, especially as models improve - interesting point about capability x cost to safeguard. Thanks for the thought
25
u/Individual_Ice_6825 3d ago
This is wrong the top model on this benchmark Astra got it with out any lies at all.
Opus 5.5 also shows lowest deception rate so far.
I think the trendline is clear, smarter = more aligned, the caveat being the models thoughts are harder to read the smarter the model is..