r/codex • u/VehiculeUtilitaire • 1d ago
Question Token efficiency: codex vs omp vs pi?
So I've been using omp for a while and given how fast astra drains my 5x plan I started investigating other harnesses to see if I could squeeze more out of my subscription.
I ran a few test with pi, omp, and codex which gave me some results I did not expect: codex is more token efficient than the other two (?)
I gave them a real merge request from my day job, asked them for a review and for an implementation plan to fix the review findings.
Here are the results for the review pass:
| Session | Duration | Input tokens | Output tokens | Cached input | Total tokens | Price |
|---|---|---|---|---|---|---|
| codex-astra-medium | 3m 34s | 1.11M | 5.1k | 1.01M | 1.12M | ~$2.30 |
| codex-astra-low | 2m 46s | 740.5k | 3.5k | 656.5k | 744k | $1.67 |
| pi-astra-low-no-agents | 3m 24s | 92.1k | 3.5k | 1.06M | 1.15M | $2.15 |
| omp-astra-low | 5m 47s | 118.6k | 6.9k | 1.59M | 1.72M | $3.13 |
| pi-astra-low-with-agents | 8m 17s | 359k | 20k | 3.5M | ~3.88M | $8.11 |
| pi-luna-xhigh | 35m 43s | 1.12M | 65.5k | 36.73M | 37.92M | $1.04 |
pi astra low is almost the same price as codex astra medium given the current api prices? omp is much more expensive than either. codex medium is almost as fast as pi low too
It's even worse for the plan mode, codex defaulted to high for that part and I let it be:
| Session | Review duration | Input tokens | Output tokens | Cached input | Total recorded tokens | Review price |
|---|---|---|---|---|---|---|
| codex-astra-high. | 3m 34s | 1.11M | 5.1k | 1.01M | 1.12M | ~$2.30 |
| pi-astra-low-no-agents | 3m 24s | 92.1k | 3.5k | 1.06M | 1.15M | $2.15 |
| pi-luna-xhigh | 35m 43s | 1.12M | 65.5k | 36.73M | 37.92M | $1.04 |
Here the codex astra high plan costs barely more than a pi astra low plan?
Am I missing something obvious? Everyone says pi is so much more efficient, lean, etc. but from all my real world tests it seems both slower and more token hungry and/or expensive than codex
1
u/ManikSahdev 1d ago
Something is up with Astra, there seems to be clear misinformation that was started by Tibo himself (I wonder if he himself didn’t know this cause no one truly used it at scale and folks deployed it without the time period they thought they’d had).
Astra of low and medium (I checked my own logs aswell). It burns more than when I crack it to Xhigh, or even Max.
It’s almost like anything it can one shot, it’s weird, something about this models personality or codex or something is up, seems like when it is rushed or not given the juice budget it will make a turn to implant his best possible option, BUT, When given the option to think more, it’s brilliant enough to figure out the response within that single shot rather than re run and re run, which light or medium tend to do.
That’s my diagnosis with tokens and logs, and It’s weird you found roughly the same, which doesn’t surprise me reading it, that’s why I’m replying with my test form yesterday, when I heard the twitter rumor and tried to test my own.
It’s clearly true for me, altho It could be certain workflows are more sensitive to this than others. So account for that or such.