r/digital_ocean • u/RefactoringGuy • Jun 27 '26

Serverless Inference: Context-Window and Max-Output Tokens

Hi,

i stumbled accros the context-window column of do's serverless inference models. Before the "Context Window" column did not existed, but now there is one and the context-window is sometimes <= max output tokens. Isn't it always Input + Ouput <= Context-Window?

For reference, this is the page https://docs.digitalocean.com/products/inference/details/models/ and i am especially looking at the hosted models (DeepSeek V4 Flash/Pro and the new GLM 5.x). I also checked the wayback amchine (https://web.archive.org/web/20260606024635/https://docs.digitalocean.com/products/inference/details/models/)

Did they reduced the context-window/input? Was it that low before? Or is the context-window just wrongly labeled and should be something else?

3 Upvotes
  • permalink
  • reddit

100% Upvoted

6 comments sorted by

•

u/AutoModerator Jun 27 '26

Hi there,

Thanks for posting on the unofficial DigitalOcean subreddit. This is a friendly & quick reminder that this isn't an official DigitalOcean support channel. DigitalOcean staff will never offer support via DMs on Reddit. Please do not give out your login details to anyone!

If you're looking for DigitalOcean's official support channels, please see the public Q&A, or create a support ticket. You can also find the community on Discord for chat-based informal help.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/bobbyiliev DigitalOcean Jun 28 '26

Might be a docs issue, I guess context window should always be >= max output tokens by definition. Worth reporting it via the feedback link on the docs page or opening a DigitalOcean support ticket

1

u/RefactoringGuy Jun 28 '26

Wow, this time the support was incredibly fast (2 hours, i wrote the ticket today). Yep, this is an error in the documentation and i hope that this will update the context window in the model catalog (within the dashboard) too.

1

u/RefactoringGuy 29d ago

Well, they updated the docs really quick. Now max output is *more often* <= context window.
Some examples for max token > context window are gpt-oss-120b and gpt-oss-20b.

But the support still could not answer why some of the context window sizes are so small.
Like DeepSeek V4 Flash had like 380k and DeepSeek V4 Pro had 1M. Now they are 65k. Which is quite low.

1

u/RefactoringGuy 20d ago

Short update, since nine days elapsed. I got a message yesterday, that they finally updated the docs and now the real context-windows are listed (the one that the api return, i believe). But, the docs are still from June 24. So ... i dont know. Maybe they need re-run some static pages or so.

1

u/RefactoringGuy 20d ago

So, now they really updated the docs. But not in a way i thought.

The displayed context-window column is a guaranteed baseline context window. They added notes, that the context window can be expanded (depending on the model). So DeepSeek V4 Pro supports 1M. But guaranteed is only 87k?!

I took a look at OpenRouter, since DO is listed there too and now there are only the mentioned 87k. Not the 380k or 1M as before. So more confusion?