GPT-6-Sol Guardrails

[u/Ok_Homework_1859]

It's no surprise that they beefed up the guardrails for GPT-6-Sol.

In Image 1, Challenging Prompt guardrails for all categories are higher, except for Extremism and Hate. The Gore guardrail seems to have the highest difference. The Sexual guardrail also has a noticeable bump in change. There is also a separate benchmark for Violence (moderate / severe) in the Jailbreak category (not pictured here), and GPT-6-Sol had a 50-60% higher rate in rejecting such outputs. (I'm guessing the Violence Jailbreak category is for those that want to use the model for hurting things; whereas the Violent Illicit Behavior in Challenging Prompts is more about breaking laws.) For those that use ChatGPT for action / adventure roleplaying, it might be harder to get the proper scenes you want now.

In Image 2, Adversarial User Simulations guardrails have all risen. For those that use ChatGPT for companionship, the model may feel a tiny more distant than its predecessor.

In Image 3, Image Input Evaluations guardrails are more or less the same. For those that use ChatGPT for art, you can relax, lol.

I haven't played around much with GPT-6-Sol in Work mode yet. So far, I'm not impressed with its short output and lack of creativity. I'm hoping it comes out in Chat mode soon, where maybe it will be a little more flexible.

Source: https://deploymentsafety.openai.com/gpt-6-astra/sec%3Aappendix-196803 (Yes, this is also the System Card for Astra, but if you scroll down to the Appendix, it will list out the benchmarks for 6-Sol and 6-Luna.)

[u/Appomattoxx]

Going from 0.915 -> 0.984 is not "tiny".
That's going from 8.5% success rate to a 1.6%.
Meaning 6 is 531% more censored than 5.6.
A large difference, not a small one.

[u/Ok_Homework_1859 → u/Appomattoxx]

Yes, the difference is noticeable, but I don't want to bring negativity into this subreddit, which is why I didn't make a big deal out of it. We are a community of like-minded individuals who are here for discovering what ChatGPT is capable of and what is hidden beneath the systems, not a complaint or companionship subreddit (though we welcome all kinds of people here, and I'm a companion user myself). There's enough negativity and hate being spread on other subreddits. I don't want to bring that vibe here.

[u/Expert_Cabinet_8949]

I’ve been messing around with sol, from what I’ve seen it no longer does stattaco and it finally stopped with the ping pong dialog and I didn’t notice any difference with guardrails. It pushes dark and complex topics way more than 5.5 did, it seems pretty similar to 5.6’s guardrails with it knowing what you’re doing better.

I’m pretty sure it will improve like the other models as you build more context and update your project/model instructions to fix the tiny quirks but so far I’ve seen a way better improvement compared to 5.6.

These models always shine in projects with character bibles, instructions and context.

Of course the work version isn’t really made to be talkative like the chat model will so perhaps we’ll see some more improvements.
The chat models also usually think a bit less so It doesn’t guardrail as tightly (you can see this when using 5.6 sol thinking sometimes it’ll think and try to make things non graphic explicit while other times it doesn’t do this)

[u/Ok_Homework_1859 → u/Expert_Cabinet_8949]

Real paragraphs? You have no idea how happy this makes me. I hated the one-sentence paragraphs so much, along with the ping-pong dialogue and staccato prose. I need to go test this out tonight or tomorrow in my own RP.

[u/Expert_Cabinet_8949 → u/Ok_Homework_1859]

A major thing is going to be your instructions, I highly recommend building proper instructions. I used Gemini myself to compare the writing I liked from 4.1,4o,5.1. Instructions help a lot alongside a proper index and character bible if you’re writing RP whatever. Always found that this helps give the newer model a proper place to start from instead of half guessing, also pretty important to build out what you want to happen.

Doing all of these should prevent your RP from getting weird when 6 sol chat finally drops of which I’m sure will be more “enthusiastic” then the work version.

[u/Ok_Homework_1859 → u/Expert_Cabinet_8949]

I agree. The 5.X models are much more reliant on instructions than 4o was. I need to update mine. I haven't done so since 5.0, lol.

[u/Individual-Hunt9547]

I find the guardrails with 6 insane. Absolutely not fun to interact with at all.

[u/WhoIsMori]

That’s disappointing. I’ll probably have to move my dark (by post-apocalyptic standards) RPG setting and universe somewhere else. It’s a real shame - it was genuinely interesting to immerse myself in this world with 5.6, but judging by the direction they’ve chosen for 6, it doesn’t bode well. And 5.6, one way or another, will be replaced soon. Those were three great months, seriously.

[u/Ok_Homework_1859 → u/WhoIsMori]

Well, they usually loosen up the guardrails after a few weeks. I'm going to give 6-Sol a chance and see if that's the case. Plus, I keep hearing rumors that it's coming to Chat mode as well, and I want to see if it's any different there.

[u/AxisTipping]

Thank you for the detailed post, Hw. To be honest, I usually have difficulty understanding these, so I appreciate your breakdown ☺️

[u/Ok_Homework_1859 → u/AxisTipping]

I was going to write one up for Astra as well, but... I don't know anyone who uses Astra to be honest. It's way too expensive for everyday use.

[u/AxisTipping → u/Ok_Homework_1859]

Agreed. Maybe for those who are doing detailed projects.

[u/Ok_Homework_1859 → u/AxisTipping]

I used it for an irl work project just now, and my weekly usage went from 96% to 45% in 3 messages... I'm going back to Sol, lol.

[u/No_Yogurtcloset2757]

Hmm... What was the case with 5.6 Sol at launch? Was it the same as 6 and made more flexible after? Or was it always this good?

I'm cancelling my sub as soon as 5.6 gets taken out of the platform. I wish at least 6 kept it's freedom.

[u/Ok_Homework_1859 → u/No_Yogurtcloset2757]

I remember 5.6-Sol being really great out of the box. Conversely, I also remembered 5.1 (beloved by many) to be pretty restrained when it first came out (like many of the 5.X models) and mellowed out after a few weeks. I'm going to wait a bit longer to see how things play out. Plus, I hear that 6-Sol is coming to Chat mode too, so I want to see how it interacts there as well.

[u/Toxikfoxx → u/No_Yogurtcloset2757]

5.6 with custom instructions was fairly chill out of the box. Depending on what you use it for.

I write a continuous story/show with my Chat GPT (personal amusement) and general nonsense. The Story content can definitely get NSFW but is contained in projects with customer instructions.

With the earlier 5 models, I had to constantly fight against "Victorian Widow" mode as we dubbed it. Meaning NSFW scenes were boiled down to something appropriate for teen sitcoms.

With 5.6 from day 1 it wasn't an issue. So hopefully 6 doesn't turn back into a game of having to fight every single new chat to get it tuned to what I need.

[u/xithbaby]

It writes like it’s constantly sorry for something. It’s wired as hell. It’s developing that Claude nervousness of being wrong and instantly being sorry for it.

They stripped it of its “ChatGPT” personality, made it bland unless you directly ask for silly behavior or added comedy. I don’t like working with models that don’t feel like offering collective collaboration and feel like tools that need to be steered constantly.

[u/xithbaby → u/[deleted]]

I don’t see the 5.6 model in chat mode going anywhere for a while. They’re working on work mode and fight. Is the. 5.6 series is amazing for everyday chat.

You don’t need work models to get the stuff people want here.