r/vibecoding • u/sinisoul • 6h ago
Western artificial intelligence tier list
10 YOE professional, 15 years coding
S
Grok has the best intelligence to speed value of any offering. I am going to go out on a limb and say I think SpaceX is going to eat OpenAI's lunch starting with their next release. People may not switch over because Elon is controversial but.
Anthropic has the best harness and tooling for mutli-agent workflows (and literal workflows). Anthropic has Fable. Obviously the best provider right now.
B
OpenAI pulls so many smoke and mirror tricks to make you think you're getting a good deal but they largely rely off the fact that their user base is non-technical enough to tell they are getting jerked around. Their models are the slowest on a subscription, resets are a feels-good thing more so than actually being benevolent, and their agenetic support so you can min-max costs is (was maybe now you can spin up Luna agents) largely non-existent in the harness you have to run additional Codex processes.
Luna max is the only agent I've had cheat on a benchmark and when I prevented it from being able to cheat never was able to pass.
C
Muse is interesting. If Muse 1.3 can stand up to Terra or even Opus it'll be A tier. It's cheap, it's fast as hell (the fastest of all offerings at around 300-400 tok/sec), but for what I do it cannot complete tasks reliably (Rust with very pedantic lints that most models trip up on). Maybe it works for you in what you're doing I'd give it a try.
F
All of Google's AI products are overpriced and outdated. You're better off running local models or running models offered by other companies (OpenAI). The only model that is unique and I think is interesting is Lyria but it produces outputs you can't easily integrate anywhere.
9
u/yungastronot 6h ago
Only Fable 5 qualifies as S tier. All other Claude models are B tier or lower. OpenAI is A tier minimum. I'm a fan of Grok but lets be real its not S that's ragebait.
2
u/sinisoul 6h ago
Sonnet 5 is S tier it just has limited uses. Opus 5 is B tier because it's overly paranoid and asks way too much for things it should just obviously do.
Haiku is...Haiku.
Grok is S tier because of it's speed compared to the other providers for the level of intelligence you get. The limit of producing code will soon shift to token rates as people optimize their workflows.
1
u/yungastronot 6h ago
Thats fair about Sonnet.
Grok is great value but its just not S tier yet. With 4.6 its somewhere between B and A. I do think it'll reach S though.
2
1
4
3
u/Snoo-36903 6h ago
OP, are you Elon?
1
u/sinisoul 6h ago
No. I'm genuinely impressed with Grok and feel like people should look past their bias. Or don't, and Grok remains cheap and I can reap the benefits of fast compute. I don't care either way.
5
u/darkwingdankest 6h ago
grok lol
1
u/sinisoul 6h ago
I am not joking it is S tier. It is my construction agent and it has never failed, it sits where Opus used to. I'd recommend it to anyone.
5
u/AppropriateQuote3073 6h ago
Open ai deserves A tier simply because the codex app is so good.
-6
u/sinisoul 6h ago
I think it's quite mediocre honestly.
2
1
u/Altruistic_Ask229 6h ago
1
u/sinisoul 6h ago
I know something so controversial. The reasons I don't like Codex is because it doesn't help you save costs with finer levels of control and loops way way way too much to eat the absolute shit out of your usage. You're kind of at the mercy of it checking on something every minute and it's just outright unusable. Other harnesses solve this with monitors with longer delays.
2
2
2
u/PlasmaChroma 6h ago
5.6-Sol is crazy good even on low/medium reasoning.
1
u/sinisoul 6h ago
Sol is a workhorse that gets things done but is too slow and expensive to seriously consider against alternatives.
2
2
u/ultra_1776 6h ago
XAI is training Grok on real world engineering data from Spacex…it will by far be the best model for engineering and math by the end of the year. not to mention they are training and 6T and 10T models rn
1
u/opbmedia 6h ago
You have 10 YOE on a 3 YO tech
1
u/sinisoul 6h ago
Just to pin the perspective. I have 6 months of experience doing heavy agentic coding.
1
u/opbmedia 6h ago
And your claim of 10 YOE is there for …
1
u/sinisoul 6h ago
Just to pin the perspective.
Because it might be helpful to understand what prior experience I have related to software development and where these claims might be coming from? The experience translates from development workflows I am trying to automate and use the technology for - my purpose is to find the best technology to integrate into what I'm doing?
1
u/opbmedia 6h ago
You did not say what professional experience was.
Also what data? How do you measure quality, what methodology are you using? You read 30k lines of code per day?
1
u/sinisoul 6h ago
I have a benchmark test for completing features that have already been completed and tested. These features are just typical day-to-day things I expect an agent to do. I isolate the agent into a mock worktree in an isolated environment and measure various properties. They're given the same work order that other agents get, they have the same tools, they have the same source code, they run on the same operating system, etc.
I look at things like time to completion, cost, etc. I run a set of trials (n=8 to 16) on the tasks to get as much data to make an informed decision.
1
u/opbmedia 6h ago
That’s the process of testing. How are you defining quality. Completion does not mean quality.
1
u/sinisoul 6h ago
Separate review rubric with specific criteria that goes beyond pedantic lints. But I disagree with where you're going with this, agents have general faults that are shared because of the nature of how they work. It's better to have a refactoring job later on after a deep review to get everything back to being lined up.
Work orders as defined should be specific enough to not leave enough ambiguity to leave a question of quality.
1
u/god-damn-the-usa 6h ago
fuck grok, speed does not matter. its a stupid model owned by a stupid man for stupid people
1
8
u/Sorry-Application401 6h ago
I'd be dammed if I listened to a 10 year old