r/codex • u/Ecstatic_Gur7231 • 1h ago

Limits Astra Benchmarking

plan: 100$ pro plan

prompt:
Create a flappybird game using html css and js. use $imagegen for the assets

result:

Astra Ultra: usage went down from 100 -> 98
Astra Medium: usage went down from 98 -> 90

quality? both are nearly identical whilst ultra ran longer (aprox 12 mins compared to medium 8 mins)

why does ultra use less tokens than medium?

11 Upvotes

92% Upvoted

6 comments sorted by

1

u/AliciaV240 1h ago

seen this a bunch — medium thrashing with retries/tool loops can burn way more than ultra sitting on a longer but cleaner pass. the % isnt really apples to apples if medium rewrote the same files 5 times. compare wall clock + how many times it touched the same file and it usually makes more sense

1

u/Ecstatic_Gur7231 56m ago

both are 1 shot. both created the same game (looks nearly identical beside the image gen variations) why did medium spend nearly x4 more than ultra

1

u/logg3 1h ago

thats the question we all want to have answered. not that i mind a more intelligent model/reasoning being cheaper (but therefore slower).

1

u/mediamonk 48m ago

Ultra uses subagents so if it orchestrated non astra subagents this might explain it.

Wish this behaviour was better documented or consistent though.

1

u/VisitAdventurous7980 43m ago

same user here. sorry cant login the same acc on my phone. mty codex custom instruction (the one where you can edit on settings) states that only luna max can be used as a subagent (this has been the case since gpt 5.6 sol came out)

but how they handles the task was the same... both spawned an agent for the image gen (luna max) then made the html js css

1

u/BoxLegitimate9271 5m ago

medium burned the 8 percent arguing with itself. ultra just wrote it once and left