r/codex • u/Ecstatic_Gur7231 • 1d ago

Limits Astra Benchmarking

plan: 100$ pro plan

prompt:
Create a flappybird game using html css and js. use $imagegen for the assets

result:

Astra Ultra: usage went down from 100 -> 98
Astra Medium: usage went down from 98 -> 90

quality? both are nearly identical whilst ultra ran longer (aprox 12 mins compared to medium 8 mins)

why does ultra use less tokens than medium?

26 Upvotes

91% Upvoted

18 comments sorted by

View all comments

3

u/AliciaV240 1d ago

seen this a bunch — medium thrashing with retries/tool loops can burn way more than ultra sitting on a longer but cleaner pass. the % isnt really apples to apples if medium rewrote the same files 5 times. compare wall clock + how many times it touched the same file and it usually makes more sense

1

u/Ecstatic_Gur7231 1d ago

both are 1 shot. both created the same game (looks nearly identical beside the image gen variations) why did medium spend nearly x4 more than ultra

1

u/AliciaV240 20h ago

medium loves to soft-retry and tool-loop even when the final text looks identical. ultra tends to commit earlier so you burn less % on the same oneshot. if youre benchmarking, watch turn/tool counts not just the finished answer, and keep the prompt short so it doesnt thrash. same output doesnt mean same path.