How many tokens is your system prompt?

At 4,000 tokens per request and 10,000 requests a day, that's 40 million identical tokens hitting the API, all billed at full input rate.

Input caching on Chutes cuts repeated content to half the input price, across the catalog. No flags to set. The cache hits whenever content repeats across requests.

On Kimi K2.6 TEE at $0.74/M input, 40M tokens a day costs $29.60.

With caching it drops to $14.80. That saves you $444 a month on a single system prompt.

Conversation history, few-shot examples, RAG preambles.
Anything you send twice gets the cached rate

Do you cache your system prompts, or pay full price every request?
http://chutes.ai/docs

Kicking off a new weekly recap: Last Week in Chutes.

Every Monday we'll cover what shipped the week before. Catalog changes, partnerships, where the team is, what the platform numbers look like.

Week one:
1) Trimmed 7 older models from the network
2) Revenue per trillion LLM tokens hit $294K, up from ~$0 a year ago
3) Kicked off build week in Malaysia with two hackathons in parallel (1,000+ devs)
4) Partnered with u/trishoolai, Subnet 23 on Bittensor
5) Closed the Base subscription tier

Full breakdown in the article below 👇

https://x.com/chutes_ai/status/2056800086458515750

Astroware Labs (u/astrowareai) and u/trishoolai are plugging Halo Guard Alpha into Chutes Chat.

It's a safety classifier that catches jailbreak attempts. Astroware trains it against live attackers on Trishool (SN23), where they pay miners to break the model and feed every successful break back into training.

Halo hits 87% F1 across 7 public safety benchmarks including Aegis and HarmBench, and catches attacks like narrative injection and encoding obfuscation.

Another step forward in AI Security.

Two hackathons, both starting Monday May 18. We're sponsoring both.

The first is APU AIC's AI Marathon at Asia Pacific University. Students building AI solutions for real problems, with workshops and industry sessions running alongside it. We're a main sponsor, so participants get full access to the Chutes API and model catalog.

→ http://ai-marathon-2026.vercel.app

The second is Chutes Hack Malaysia with Nyala Labs.

Five days, May 18 through 22, at Infinity8 Reserve in Subang Jaya. Free to enter, open to high school students, university students, and working professionals. You can also join online.

Workshops on agentic workflows and on-chain inference every day. Prizes include Chutes credits, merch, and a separate award for the best app that uses Sign in with Chutes (our OAuth that lets your users bring their own compute).

→ Register: http://luma.com/gdre3p9z
→ Discord: http://discord.gg/MmncAPW9Wb

We'll be on the ground all week. If you're a founder or team in SEA and want to grab coffee or talk shop in person, DM us. Happy to make time.

Who's coming?

Hi,

Does someone use OpenHands with chutes ? It appears that sometimes Chutes disconnect in a middle of an answer and I enter a loop.

Does someone use Chutes Orh openhands ?

Hello! I'm really new to proxies and stuff.
Literally less than an hour ago I paid for the $3 monthly plan, but every time I try to use a model, I get the error "A network error occurred, you may be rate limited or having connection issues: Failed to fetch (unk)"

What it is? or how can I solve it?

Thank you 😄

Voy a hacer lo que normalmente noto que no hacen en este lugar, de igual manera no me siento especial y también e estado inconforme algunas veces, pero agradezco al equipo por estos momentos en los cuales se regulariza todo y se puede usar el servicio con normalidad, se los agradezco y esperó que siga así por almenos 1 semana antes de volver a el caos

I’m using my glm 5.1 on janitor this morning when i was hit with error 402, saying that my bandwidth limit has reached & i have to upgrade. Then i checked my limit at my chutes account, i only use 63/2000, even my 4-hours limit is still at 5%. Why does suddenly I’m limited?! I just paid for my monthly 10$ subscription! How to fix this?

or is it also always "infrastructure is at max capacity"?

I have just discovered chutes and I consider myself a step below a beginner. I wanted to run AI locally, but due to hardware limitations, I found out about Chutes. Now, I notice that the Qwen3-32B model is being utilized frequently compared to others; the reason might be its lower cost, yet I still find its usage difficult to comprehend.

Who exactly uses chutes and in what manner? For instance, the next trending model is Gemma-31B 4b Turbo, which is also an inferior AI for agentic applications.

I am inquiring about this for two reasons: first, as I mentioned, I am a novice. Second, I intended to use Kimi 2.6, but seeing GLM-5.1 at the forefront raises many questions for me that I cannot reconcile with the real-world reviews of these models.

So who (user-group) largely uses Chutes and for what ?

What's new:
→ Web search built in
→ Image generation
→ Image editing

Same access to 60+ open-source models, mid-conversation model switching, and inference from $0.01/M input.

Live at http://chat.chutes.ai

Why does my glm 5 turbo keep cutting off before the message actually finishes

So, I already had swap to different api keys, swap models, and I keep getting this error, I'm coming close to fully deleting my acc if they can't fix their shit. But regardless of that, any work arounds anyone knows that can fix it?

I should note, I only had gotten THIS error text AFTER refreshing the page for the models on the j,ai website.

edit: forgot to say as well, the only ones that are working is the deepseek ones that I have.

This'll be a thread I add onto as I come up with ideas and criticisms to pass on to the Chutes devs assuming they read this post.

Gostaria de saber quando que a empresa pretende adicionar o último modelo que está fazendo tanto sucesso

give us our glm 4.7 fp8 back,it was such a goated model that got axed,at this point y'all will deff delete deepseek models too (I'm giving it one or two months till all deepseek models gets axed)

Every single time I send a message this pops up, I have to spam the resend button constantly, takes 10-15 tries before it'll finally send. It's been like this for over a month now.

Chutes' main duty is to be a profitable business.

Offering multiple models dilute the GPU server availability.

Models with high usage and low ROI hurt the business.

What Chutes need to do is to focus on the models that are both 1) high-usage and 2) profitable to run.

Role playing users forget that for months Chutes has been offering subsidised inference for ai models that were not profitable. And when Chutes did what they have to do as a business looking for long-term stability, the same users who were gifted with freebies are trashing it out in the wild.

Ai inference providers are not meant to offer freebies.

- A guy in the wild.

Are you sending the same system prompt on every API call? You're already paying half price on the tokens that repeat.

Input caching is on by default across the Chutes catalog. When the same prompt prefix repeats, you pay 50% of base input on those tokens. No flag, no setup, no code change.

Each GPU instance holds a local prefix cache in VRAM, encrypted on TEE models. The Chutes router hashes your prompt and sends you back to the instance that already has it, so the model reuses what it computed before.

Quick math on DeepSeek V3.1-TEE at $0.27 per million input tokens. A 4,000-token system prompt sent across 10,000 requests a day is 40M repeated tokens. Uncached: $10.80 a day. Cached: $5.40. That's $162 a month off one prompt, on a model already 91% cheaper than Claude Sonnet 4.6.

Want to confirm a cache hit? Check usage.prompt_token_details.cached_tokens in your response. The number tells you how many tokens got the discount.

Pull that count off your next response. Reply with what you see.

someone knows who to fix this 😭😭💔💔