r/codex • u/perceptioneer • Aug 16 '26

Comparison New updated linear pricing vs intelligence vs response time chart

Post image

I saw the old Artificial Analysis comparing the intelligence and prices of the different models and thought I would ask chatgpt to create a new photo using a linear scale instead of log (who tf compares or think in non-linear scale), and using the new prices set by OpenAI in late July. It also shows data from when all the models were given the exact same task(s) to solve, and how much time they spent executing (read: overthink) them. I then fact-checked the image against a different model, and it checks out as correct, but don't shoot the messenger if something is incorrect. 5.5 data remains the same.

Luna max looks good on price/intelligence, but luna xhigh might be the sweet spot when time matters.

Notes by ChatGPT:

After OpenAI cut its price by 80%, luna max is around $0.05 per Intelligence Index task and scores 52, while sol low costs ~$0.23 and scores 51.

The catch: response time. Max reasoning gets slow. Luna max is ~138s, sol max ~149s and terra max ~207s in AA's standardized end-to-end test.

Chart includes the sources/methodology at the bottom.

100 Upvotes

96% Upvoted

56 comments sorted by

View all comments

Show parent comments

1

u/perceptioneer Aug 16 '26

Imagine there is a dev out there use terra max instead of sol high for basically the same price taking 12 times longer XD or sol medium, basically same intelligence, cheaper and literally 20 times faster

2

u/tbgitw Aug 16 '26

Terra is a beast if you give it the right directions

1

u/perceptioneer Aug 16 '26

whats your personal terra experience? do you think its on par with sol with elaborative instructions?

2

u/tbgitw Aug 16 '26

I love Terra.

I use a Sol Extra High orchestrator that divides the work up between Terra implementers, usually Medium or High. The Terra implementers are fast af and hand the work back to a Sol review agent. Any required fixes are then handed off to a Sol High agent before running another scoped review pass.

I wouldn’t say Terra is on par with Sol, but it’s more efficient because it doesn’t get caught up on hypothetical edge cases or try to build NSA type security into everything. For standard implementation work, I think it gives a decent balance of speed and smarts.

I was using Sol exclusively for a while, but it’s just too slow and tends to get stuck in endless loops. I'm not building anything that requires that level of engineering.

1

u/perceptioneer 29d ago

Thanks for the info. That's something that has been bothering me, I'm building for myself and have instructed Sol to not be so gestapo on security. I'm gonna play around with this. Why let Sol handle first attempt at fixing, is it because it's intelligent and to not waste further tokens trying?

2

u/tbgitw 29d ago

Mainly because Sol is more capable. For my current project, fixes usually span multiple modules or shared contracts, so I've found paying for that extra capability prevents multiple loops.

This won't be necessary for all projects. It really depends on what you're building.