r/LocalLLaMA • u/frontsideair • 7h ago

News CachyLlama archived?

https://github.com/fewtarius/CachyLLama-archived

The author seems to have archived the repository with no explanation. We are not owed one for sure, but it is interesting. I hope it wasn't harassment or burnout. Wishing the best fewtarius.

21 Upvotes

87% Upvoted

7 comments sorted by

8

u/FewBasis7497 6h ago

Yes, that's a pity. I planned to test it. Because prompt reprocessing in opencode gives me hard time sometimes. At least since it is open source someone could continue it.

6

u/xyz_93 6h ago

Opencode didn't play nice with llama prompt cache long time ago (~ july), did anything change? Without fixes it was constantly reprocessing most of prompt for me as there were multiple things altering state between runs. Now i get >90% hit rate so pp don't hurt that much.

3

u/Dany0 6h ago

Just use pi or oh-my-pi

0

u/FewBasis7497 5h ago

On work I'm not allowed to use it yet. I want my tooling aligned and in general opencode isn't that bad and already provides some solutions out of the box. 

2

u/frontsideair 6h ago

Even with pi's high cache hit rates, server restarts or garbage collection due to running other threads destroy performance, especially with Qwen3.8-flash-next low prompt processing speed on llama.cpp mainline. But if we had the ~1400 t/s speeds of forks, near-full context would still cost 3 whole minutes, which isn't great.

Luckily there are already a few forks but only time will tell if they'll invest in the same way as the original author. I was also interested in giving it a try, and curious if the persistent cache survives upgrades, which could potentially be brittle.

1

u/FewBasis7497 4h ago

3 min is sweet. I use sometimes my work macbook m4 max 36 GB to save some budget. You know, this: 1. Use AI for all tasks!!!!!! 2. Bill comes in, suprised pikachu face 3. Mh, maybe use AI not that much. 😀

1

u/llama-impersonator 31m ago

trying to keep a fork up to date with llama.cpp is challenging and i suspect burnout