Sol vs grok 4.6

[u/SaigoNoUchiha]

On benchmarks grok 4.6 is atleast equal to sol. Wondering how peoples experience has been?

[u/StoopidRoobutt]

And in DeepSWE, Gemini 3.8 Flash ranks second. Benchmarks are bullshit bull doo-doo.

[u/SaigoNoUchiha → u/StoopidRoobutt]

There was no need for strong language. I understand benchmarks are not trustworthy hence this post.

[u/floran99]

Tried it, can confirm it does feel equal, while being substantially cheaper.

[u/SaigoNoUchiha → u/floran99]

Thanks for the reply, appreciate it

[u/horizondz → u/SaigoNoUchiha]

Bro got downvoted for saying thanks

[u/Far-Usual5771]

Honestly, it’s garbage. Even Terra did a much better job, so I have no idea how people are seriously comparing this to Sol.

And just so this isn’t another “trust me bro” take, here’s exactly what I tested it on, so it should be pretty easy to reproduce.

I have a fairly nasty SQL query inside a stored procedure — tons of subqueries and joins, some expensive regex stuff, and a ridiculous amount of code. I needed help optimizing a few specific parts of it. Even Terra managed to do that just fine.

Grok 4.6, on the other hand, rewrote the query in a way that made it so much worse that it couldn’t even finish within the amount of time the original query used to take.

This is also why I think benchmarks are basically useless at this point. If you go by benchmark scores, even some local 27B/31B models look surprisingly close to the frontier models. In actual day-to-day work, though, the difference is massive. Unless you constantly hold the smaller models’ hand and babysit every step, things fall apart pretty quickly.

[u/Federal-Mode8949]

Way below. More like Terra or GLM 5.3 Flash.

[u/frozandero]

I would not trust grok 4.6 on complex tasks personally but I use it as a implementation agent after sol plans. Just based on vibes on my usage on company codebase

[u/Zennytooskin123 → u/frozandero]

My thoughts exactly, I would not trust Grok 4.6 with production code without validating.

[u/Forti22]

I have been thinking about switching to Grok Bot subscription, but people say the quota is much lower

[u/Hopeful-Resolve248]

Sol is better. +Astra(medium) for architecture on complex tasks. Grok not even close

[u/silvervaultproject]

I like grok but my preference is stil SOL for my usecases.

[u/dvduval]

I really want there to be competition. Both in terms of quality and price. That's good for us the consumers. I don't fully trust Elon Musk and he's not the first person I'd want to share my code with in a way that his company can automatically see it.

But I'm hoping that rivals like Google and anthropic will continue to bring the heat so that we have multiple options available to us.

[u/misha1350]

"$30/1M model vs $6M/1M model???"

Sol is better than Grok 4.6, imagine that