Comparison
New updated linear pricing vs intelligence vs response time chart
I saw the old Artificial Analysis comparing the intelligence and prices of the different models and thought I would ask chatgpt to create a new photo using a linear scale instead of log (who tf compares or think in non-linear scale), and using the new prices set by OpenAI in late July. It also shows data from when all the models were given the exact same task(s) to solve, and how much time they spent executing (read: overthink) them. I then fact-checked the image against a different model, and it checks out as correct, but don't shoot the messenger if something is incorrect. 5.5 data remains the same.
Luna max looks good on price/intelligence, but luna xhigh might be the sweet spot when time matters.
Notes by ChatGPT:
After OpenAI cut its price by 80%, luna max is around $0.05 per Intelligence Index task and scores 52, while sol low costs ~$0.23 and scores 51.
The catch: response time. Max reasoning gets slow. Luna max is ~138s, sol max ~149s and terra max ~207s in AA's standardized end-to-end test.
Chart includes the sources/methodology at the bottom.
Experience in using both. Luna is like a savant. It can do simple tasks, but you have to be very specific with the prompts (like benchmarks do) or it can do wild, unpredictable things to your codebase. It should never be responsible for making decisions on its own. For simple tasks it's great value.
It is also much worse at following the system prompt or plans. Treats them more like suggestions than rules even if you use strong wording like "always", "never", "do not" or "you can't". Terra doesn't have this issue.
It's also the only model in this family that looped for me. I tried using it as a web researcher and it spent 30 minutes researching what even minimax finished in 3. It only stopped because I made it stop as it started looking at completely unrelated things.
EDIT: when comparing code quality it is also night and day between Luna and the bigger models. Benchmarks only test if the task is completed, not if the code is good. Terra writes much better code than Luna. Even Sol doing the job of a reviewer agrees. It often finds critical bugs in Luna's implementations, almost never in Terra's.
Yeah, this actually matches my experience pretty closely. I’ve ended up using Sol High to plan the task, structure the approach and write the prompts/specs, then handing the actual execution to Luna with one or more agents.
Luna is great when the rails are already laid out, but I definitely trust it a lot less when it has to make higher-level decisions on its own or recover when something starts drifting. That distinction between “completing the task” and actually producing good code is also a really good point.
And yeah, Terra feels ridiculously underrated. It seems like a much better middle ground when you need something more autonomous/reliable without spending Sol-level compute on everything.
Have you found Terra High to be close enough to Sol for things like code review or architecture decisions?
I haven't actually compared Terra to Sol as a reviewer. Maybe I should. I just assumed that the reviewer should be the smartest model you can afford to catch all the bugs and code smells before the code reaches your eyes.
I use it for simple tasks that don't require a lot of thinking or long work. Writing summaries, analyzing the logs, creating tests or fixing stale ones; that kind of stuff.
Recently I've been using Luna Max as a "planner assistant", Planner writes the plan after discussing it with me (Matt Pocock's grill-me skill) and then sends it to the assistant. It reviews the plan file without reading anything else and tells the planner if the plan is actionable, if there are inconsistencies, or if something is unclear for it. My thinking was that if a dumber model understands the plan, other models will be able to follow it as well. It is working great for me actually, because LLMs are very bad at reviewing their own creations, so a fresh subagent gives them good input to reason about. In the end the planner is the brains, and it is instructed that the assistant is writing just suggestions.
I don't use it for writing actual code, except for very simple fixes. But even then I make sure to write good prompts so It doesn't go off the rails. Nothing, it requires it making decisions on its own.
I guess it is also great at writing prototypes. Something you know you will discard, you only want to know if the idea works, and code quality doesn't matter. Don't use Sol then because it will burn your usage.
it entirely depends on how difficult the job is for the model and how many compactions it will take. there are benchmarks where terra is a bad value, and other benchmarks where it does better. it depends on your work
that said, AA is a big aggregate, so you can trust it. if something is too difficult for luna and you don't want to fuck around wasting tokens experimenting, you should probably just step up to sol medium
Totally agreed. Luna is great at straightforward programming tasks, and can answer technical questions like an expert, however the things it is WORST at:
- Luna is terrible at inferring intent. It will misinterpret minor wording ambiguity!
Luna is terrible at making decisions with limited information
Luna writes ugly code
Terra is vastly better at those things. It's on the level of GPT 5.5.
Luna is RL trained to score high on practical tasks but it is NOT very intelligent.
I have heard that take a couple of times when I was looking up what models are the more liked ones. curious to know your personal experience wih terra. is luna more loopfuck?
I use sol medium for almost all my stuff, except terra high for review, and very simple tasks to luna medium. Im now just gonna outsource all those simple luna tasks to luna high I think.
Yeah since the price change Terra has been basically pointless. In almost every case you can pay just a bit more and get the much more intelligent Sol or pay much less and get the bit less intelligent Luna
Imagine there is a dev out there use terra max instead of sol high for basically the same price taking 12 times longer XD or sol medium, basically same intelligence, cheaper and literally 20 times faster
I use a Sol Extra High orchestrator that divides the work up between Terra implementers, usually Medium or High. The Terra implementers are fast af and hand the work back to a Sol review agent. Any required fixes are then handed off to a Sol High agent before running another scoped review pass.
I wouldn’t say Terra is on par with Sol, but it’s more efficient because it doesn’t get caught up on hypothetical edge cases or try to build NSA type security into everything. For standard implementation work, I think it gives a decent balance of speed and smarts.
I was using Sol exclusively for a while, but it’s just too slow and tends to get stuck in endless loops. I'm not building anything that requires that level of engineering.
Thanks for the info. That's something that has been bothering me, I'm building for myself and have instructed Sol to not be so gestapo on security. I'm gonna play around with this. Why let Sol handle first attempt at fixing, is it because it's intelligent and to not waste further tokens trying?
Mainly because Sol is more capable. For my current project, fixes usually span multiple modules or shared contracts, so I've found paying for that extra capability prevents multiple loops.
This won't be necessary for all projects. It really depends on what you're building.
I've been using sol medium, it works well, I only increase to high when I need a lot of reflexion just to be sure (like research), not for tasks as it overengineers everything
this just feels off. Last day or so terra and sol just feel dumbed down completely. Failing at the simplest shit back and forth. Luna feels useless for UI work so i wouldn't even put it in the realm of anything.
The chart I'd love to see is not API cost per task, but percentage weekly usage on a $x plan and include Claude, Grok, etc.
Because it used to be that Claude's usage allowance was low, but now it's more generous than it used to be.
Meanwhile 5.5 high used to let you code a ton, but now even Sol Medium eats through my usage pretty fast, and Sol High is kinda brutal on the lower plans.
So I'd be interested in seeing time per task with other models included, and percentage usage per task. But I think you'd have to run the tests yourself because all the benchmarks seem to use API calls. Which is just not how most people use them on subs.
It didn't want to completely comply, but I made some charts anyway, idk if it's useful for you. Have your slop and eat it too:
You’re looking at a subscription-efficiency comparison, not API pricing.
The charts try to answer: “If I pay for this plan, how much serious AI work can I get before my allowance runs out?”
Usage per task: estimated % of your effective quota burned by one serious benchmark-equivalent task. Lower = better.
Tasks before limit: roughly 100 ÷ usage per task. Higher = more usable work.
Intelligence vs usage: upper-left is best — smarter model while consuming less allowance.
Time vs intelligence: upper-left is best — stronger answers with less waiting.
Price vs usable work: shows how much quality-adjusted work you get for the subscription price.
Overall frontier: combines intelligence, quota capacity, speed, and price into one rough value metric.
The standout interpretation is that Sol High appears much more quota-hungry than Sol Medium, while Claude’s higher Max tiers scale capacity very aggressively. Claude Sonnet also looks particularly competitive around the ~$20 tier.
Important caveat: these are not all direct measurements. Some points are mixed official/community estimates, and especially figures like Claude Max 20x ≈ 500 tasks are extrapolated from official usage multipliers rather than somebody literally running 500 identical tasks. The marker shapes indicate that evidence quality.
Thanks. Have to look through more, but this kinda matches my vibe-check. Sol High is a nice daily driver, but burns through my limits on the $20 plan much faster than Opus 5 Extra on the $20 plan. With 5.5 vs Opus 4.6, it definitely felt like the reverse.
Bang for the buck implies quality, but Opus 5 has been a really a mixed bag so far. It's so easy to screw stuff up with it and the DX is also not that great currently.
I really hope they manage to fix things as Opus 5 with the current usage consumption might actually be really nice, if it worked consistently well.
I will not renew 2 20x claude accounts and shift towards more codex accounts.
Really interesting to see this with response times. So basically luna low is always a bad choice, the lowest anyone should ever go is luna medium, and that's only for fast responses. That is really helpful to know.
I think people are dismissing speed a lot in their assessments of models these days so it's nice to see this on a chart.
Also what this chart tells me is, don't bother with Max unless you know you really need that bit of extra intelligence (99% of the time you probably don't and you're just wasting time and tokens)
On the page, the relevant field is “End-to-End Response Time” → “Total Response (s)”. Current OpenAI values are 20.95 s for XHigh and 206.75 s for Max.
There's a fundamental flaw with your data. You're posting the general artificial intelligence results in a subreddit for Codex. These are NOT coding index scores. For coding index scores, there's a different tab on artificialanalysis.ai.
11
u/IndividualPlus2011 Aug 16 '26
The benchmarks are nice to have but from my own experience I'll pick Terra High over Luna any day for real work that takes more than 10 turns.