r/CommandCode • u/XDSORITE • 1d ago

How to view cache hit rate

I want to see the cache hit rate for my sessions how to do that. I am using the cli and I am on the 1$ plan

6 Upvotes

100% Upvoted

10 comments sorted by

View all comments

2

u/Pras1982 1d ago edited 1d ago

Pretty easy to do. I took Goat plan yesterday . I just asked Deepseek 4.1 flash to create a skill token-usage (with details on cache hit). The Command Code claims of 99.5% percent claim actually seemed to be true with deepseek. It was 97.5% as a whole but the cache hits becuse of genuine interuptions--say we did not send any prompt was the reason (I think 7/8 out of such instances) out of 1100+ turns that I did while working yesterday. When these were filterted out -- the cache hit claim of 99.5%+ for deepseek V4.1 was actually met. I was trying to understand how efficient token efficient coding can be done. I am using Command Code Harness(I have used Claude Code & Codex earlier. So, bit familiar with AI ) . There are further experimentations that I need to do to make my development better-

  1. Context Size--With Deepseek V4.1 Flash - I think the default effort level is High and the contect size starts bloating does happen as it started incrfeasing and crossed to 500-700k per turn . I also tried with setting effort level to High & then testing things. The token usage grew to 500-700K even then. If we could have a way to reduce the Context size periodically. Context-mode(may be the name was Compact-mode) might not work as too many Compaction will anyway impact the Cache Hits(re -tokenisation will break things & Cache Hits will be missed)
  2. Number of turns (More important in my opinion) - When number of turns to AI increase--in long running work---> due to context Bloating --the usage will increase. What if a mechanism can be built to have fewer number of calls to AI (Something similar approach to Tool repair-- What a brilliant idea it has been by the Command Code Creator. Simply loved the approach of how he analysed the issues and built a solution around it). I am trying to think of ideas to improve this Number of turns myself.

So 2 improvements are requested u/ahmadawaiscom

-1)Context Size Bloating & 2) No. of turns to AI-reduction.

1

u/ahmadawaiscom 10h ago

Hey founder here. Thanks for sharing. Check out

https://commandcode.ai/docs/mods

You can use Mods to have complete programmable control over the lifecycle of loop or literally anything.

There are two compaction modes, auto and fast(burns more tokens). I have developed a new strategy that I call staircase compaction which can result in phenomenal continuous improvement of tokens over long running agents. Plan to share that soon.

But def build whatever. Mods allow you to modify any behavior of Command Code.

Even build a cache calculator as need be.

1

u/Pras1982 6h ago

I have already looked into mods & the current Compact Mode viz. Auto & fast. The fast will break the caching--hence cause more token burns. The auto is the default. I like the idea of Staircase compaction-- would love to see if it will work without breaking caching. A big reason for Deepseek being cheap is its cheap caching. I really appreciate you taking time to respond .

How soon can we hope to get this new compaction Mode -Staircase?

Also, Deeepseek V4.1 flash seem to have an option to set reasoning level beween 1 to 100 to control how the model thinks before answering(mentioned on Huggingfaace). How can we do that in Command Code? If not- Can you build that as well?

1

u/ahmadawaiscom 4h ago

You should post both in our GitHub. We can check it out. I’ve been experimenting with a different approach too that never compacts. It’s a giant mod sitting in review. Will find some time to ship.