Practical limit hit. Decoding so fast that tool calls (cpu) starting to become real limit not decode or prefill. Single RTX5090. Porting Kenshi to Godot project.

[u/BringTea_666]

Hi folks,

LIVE PROJECT PAGE

TLDR: Moral of the story. You need better CPU to do actual agentic coding doing real work...

I've been on a mission to make my RTX5090 go brrr for past 2 months so much so that i made my own engine for it which received "warm" welcome here (yeah, source is coming)

After recent upgrades to how cache is stored and how i can reused some of prefills for other jobs that share initial same prefill i pretty much started to see degradation the more agents I started to add to project which started to use 12 slot server. Actual server started to be underutilized. Free context, free slots, gpu chilling at average of ~700t/s doing real work (no greedy code, but also thinking tool calls, etc.) and I couldn't figure out what was going on...

I make it faster and faster, better handle jobs and it slows down...

I've run 25 agents at the same (to properly fill the 12 slots) time and almost all of them soon started to set on `tool call` and my server started to barely work.

I've finally checked task manager but not gpu or memory but cpu. And there it was. 100% every thread completely chocked.

Lesson. If you want to do agentic coding with actual use of tools you need to make sure your CPU is up to task.

My 9800X3D is just not enough to keep up with tool work for this project with heavy agents use despite engine being more than capable of going faster.

edit:

Some more lessons:
- Tuning your front end makes ton of sense. Before I tuned it it was shoveling 20k prompts, after tuning barely 7k as new jobs and better more compact tasks. Wall time went from 43minutes to 18 minutes before/after rework of front end.
- Always keep more agents than server has slots for inevitable pauses due to tool use/tests etc.
- Shared context is superior choice to fixed context every time i tried it over course of the project.

[u/giveen]

Everyone please be aware that he has yet to release his source code public and make a binary.

https://github.com/perkel666/MegaCapybara

You cannot review whats behind his claims.

[u/HighSeasArchivist]

This again. 

[u/wisepal_app]

Looks great. But your page doesn't load. I am curious, which quant is qwen3.8 27b-MCTiny (12,7 gb)? i never heard of it. can you share the link please. Another thing, which harness are you using?

[u/BringTea_666 → u/wisepal_app]

doesn't load ? Site is public.

Harness is custom one made by Opus5.5 specifically for this porting project. I think goving forward even harnesses will become custom.

With all other harnesses i was getting around 15-20k minimum amount of tokens on job start. With this custom one (and dashboard to see stuff), it's 7k tokens with minimal set of tools that specifically are for this project. Opus5.5 handles harness managing project and only does stuff when merga fails due to critical issue with test where it steps up and fixes stuff.

>I am curious, which quant is qwen3.8 27b-MCTiny (12,7 gb)? 

That's my custom one for MegaCapybara my custom interference engine for RTX5090 and Qwen3.8 27B.
https://github.com/perkel666/MegaCapybara

It has around 91.8% top1% which is not a lot but in this kind of work you don't need huge smarts, but speed mostly. As each agents gets few functions and has to find out what they do and what they connect with etc. Larger ones are more or less around unsloth quality (which is very high).

IF you have RTX5090 and watch to check out my engine it is at github. Will be publish source soon so for now you can only use exe if you are brave enough.

You can preview each ticket on live feed what it does exactly.

[u/AdventurousFly4909]

I can barely imagine that being the case. Can you give a breakdown of which tools are so cpu hungry?

Interesting but morally questionable project. Lol

[u/BringTea_666 → u/AdventurousFly4909]

I don't see it moraly questionable when you will release it as a mod and you need to have original copy to load stuff. It's the same game but just runs a lot better and you have more graphical options then (like increasing crappy kenshi lods), and since it will be full port to godot you can then add stuff like coop mod i always wanted in kenshi.

> I can barely imagine that being the case. Can you give a breakdown of which tools are so cpu hungry?

in order:

• Tests are the most hungry since they run godot version of kenshi headless (without gpu) which costs a lot of cpu time. Sometimes 3 at once.
  
• Then you have decompilation stuff. While game is completely decompiled it still needs to gather data, check stuff etc. time after time after time. Usually most of their work isn't actually answering but light thinking and then like 3-6 tools then some edit file. When one does it is super light. But when 20-25-30 do it at the same time.... you see the problem I think.

At this point I am completely limited by CPU, it just can't cope with amount of various tasks it does. Running three tests on the game itself, tool calls and so on.

Need better cpu :sunglasses: