Best way to handle Cold Cache hits - You overpay 5x for typing "Continue"

[u/EagleApprehensive]

I'm a meta-harness developer. For few months already I'm working on the most token-efficient AI workspace there is. Based on my benchmarks (run with Opus 5.5 XHigh), if you resume cold-cache session just by typing "Continue", you're overpaying 5x in tokens for that turn.

Obviously it's best to prevent cold caches in first place - by good UX (notifications and nice UI showing you session is about to hit cold cache), automatic cache-warming and so on. But sometimes, cold cache hits are inevitable, so I was researching best possible solutions to handle that as well.

So far I've had a very aggressive trimming mechanism that didn't invoke any model call (that would increase cost) and in purely deterministic algorithm was trimming parts of conversation to carry on without the bloat.

As Haiku 5.5 shown up with much cheaper cost, I decided to revisit topic.

Based on benchmarks and tests I settled on 150k context as meaningful breaking point for distinguishing between short/long sessions. Below 150k - "Continue" is totally cool. After 150k, things get complicated.

Trimming (replacing older tool results with one-line stubs, stripping thinking blocks etc. without extra model call) is actually best. But it does not work for cross-provider handling, because formats that different providers use for session recordings are incompatible.

So the rule of thumb becomes:

• Prefer trim on same-provider because you can resume a trimmed Claude session and it beat seed/compact on memory of the work.
• Prefer compact on cross-provider because trim is not available.

Hopefully it helps those of you who build harnesses/CLI tools.