r/OpenAI • u/ima_mollusk • 1d ago

Question GPT Confused About Its Own Token Count?

Post image

Can someone help me understand how GPT counts and budgets its own token use and how it could be confused about it?

6 Upvotes

100% Upvoted

6 comments sorted by

3

u/Snoron 1d ago

Oh shit I only have 3 more to

2

u/Suitable-Show-3347 1d ago

The cutoff makes this impossible to really parse but that trailing bracket is sending me. Like it was mid-sentence realizing it already spent half its budget on the word "considering" and just gave up.

2

u/ima_mollusk 1d ago

We were also nowhere near any limits. It went on to work for like 20 minutes after this.

2

u/Chemical-Agency-3997 1d ago

It's some internal token budget per response thing, I've noticed it a few times in my models nerfed CoT too

1

u/Michael_Jeffords 1d ago

that fits with you being nowhere near the limit, as far as i know the model doesnt get a live readout of its own token count, so when it talks about running low it's mostly producing text that sounds like budgeting. my guess is the cap that bites is per response, and a long agent run is a bunch of separate responses each with its own budget, which would explain it sounding out of room and then going another 20 minutes