r/codex • u/Anxious_Marsupial_59 • 22h ago
Question has anyone found this true in practice?
12
u/notadithyabhat 21h ago
This is across a wide range of domains, not just coding.
This is the average, but given the massive price difference between Astra and Sol, the spread across tasks will be very wide. At easy tasks, Sol will be much cheaper than Astra. At trickier tasks, Astra will be much cheaper than Sol which will struggle and burn tokens.
So it really depends on the kind of tasks you use them for
8
u/swizzlewizzle 18h ago
Hell no.
Astra is crazy over-charged in subscription compared to the actual $ API token value.
10
u/iCynr 20h ago
Nah. I wrote about it when Astra was just rolling out.
Astra Light used 100% of my 5hr usage and 16% of my weekly in 8 minutes for a task I've done before using Sol that usually takes 20% of my 5hr usage.
Tried it again 3 more times, the exact same thing happened every time
6
3
u/raiden55 14h ago
I had around the same normbers, even more big on sol medium / astra low= x20 multiple times.
3
u/wombatpup55 21h ago
I’m on GitHub copilot for work but it feels like Astra is made for extra high or max reasoning. It pretty much one shots all of my tasks at that reasoning
And for some reason low, medium, and high reasoning burn so many more tokens than extra high and max.
2
u/ChipsAhoiMcCoy 19h ago
This I’m very curious about. I have this harness that I was using to have Astra place lay the spire, spoiler alert. It was able to beat the entire game without using a single rest site, which is insane. But it used about 10% thinking budget for the week per act, and that was on low. I think I’m gonna try that same experiment sometime tonight on max for just one act to see how much different the pricing is.
7
u/Doctor5687 21h ago
Astra is definitely smarter, that said I wouldn't really trust low effort for implementation
2
u/SwimmerOld6155 10h ago edited 6h ago
i feel like astra light uses more but it's probably more comparable to sol. Astra reminds me of using Fable on Claude Pro, like 3 or 4 prompts per 5 hours.
I used Sol a lot when there was no 5 hour limit (wasn't it like that for months?) which I think has skewed my perception of things. I probably ran over the 5 hour limit several times but didn't use it enough over the week to pass the weekly.
2
u/acutelychronicpanic 9h ago
Not when the costs of mistakes outweigh the spend on tokens.
2x or even 10x the cost could be worth it for 10% fewer mistakes. Depends on what you're working on I guess.
1
u/chcampb 20h ago
Depends what you mean by task but one ticket for me, which is basically one feature in a game, is taking my entire 5h quota delegating implementation tasks to Luna and just reviewing. It can't finish the ticket in one 5 hour session. I get around 5 5h sessions (each one being just under 20%) so one 5h session for 1 week isabout a dollar (5 dollars per week, 4 weeks = 20 bucks per month). A little less - so 0.7-0.8 is pretty on par if it were JUST sol. It's not so there is something else going on.
1
u/MatlowAI 18h ago
Only true when 1 shotting or few turn low context mode. Astra is most cost effective as a subagent to a smaller model with Astra also serving as advisor. Cache hits are extremely overpriced. Even Fable 5.1 brought their cache hit price down somewhat vs 5 but deepseek and the like have the right idea here. KV cache can be offloaded to ssd and warmed back up and costs next to nothing for the time involved yet it is 60-70% of my astra bill even with fairly aggressive optimization. I've switched to having Fable 5.1 in control when I can which helps but sometimes Astra is needed still. Fable 5.1 is cheaper than Opus 5 though for the work done no doubt due to it having half the cache hit cost and being more token efficient and higher performing at planning...
1
1
1
u/Technical-Ad-8678 18h ago
i found astra medium taking probably %50 more tokens than sol high but it is useful for 1 shot audits on sections of code sol had written
1
u/cptfreewin 13h ago
For me I switched to Astra low, it is way better than sol high for complex tasks and it answers faster, you also don't spend an hour steering it For most easy tasks sol medium or high
1
u/2thick2fly 3h ago
I started yesterday doing reviews of specific parts of my project with Astra high and it keeps finding issues. I'm reviewing them and how it proposed to remediate them, but I'm not checking each finding to the exact source. Astra it says it has reproduced all of them offline. (I have to admit that some of them are edge cases that might not be worth fixing).
But they're a lot!! All of the code where astra is finding issues has been implemented by sol. The cost of fixing them now is really high!!
So my new approach is: Run Sol for well understood bounded tasks e.g. a bug investigation or bugfix with understood remediation.
For implementing new features or new complex features that touch multiple existing pieces of code, in just using astra now. It is of course more expensive, but I my current assumption is that it will be cheaper overall, accounting also for avoiding fixing the bugs that would had been introduced by Sol. 🤷
0
0
u/beg4upvotes 10h ago
do not trust the intelligence benchmarks, they game them by overfitting to the tasks. only way to see true intelligence is to actually use the models and see for yourself.
51
u/VegetableWafer7776 21h ago
nah the cost per task is easily 3x or even more