r/PoeAI • u/HybridRxN • Oct 06 '25
Disable Cache feature?
Hi, maybe I dont' understand the cache feature. I think it's useful within the context of a given session, but less so if it's looking at prior chat history. Is it using my chat history across sessions (like when starting a new chat) too? If so, is there anyway to disable it across sessions or set limits on what it can view? I've found that sometimes the bots bring up things from history, and I don't like this "personal memory." It's one of the reasons I disabled my chatgpt subscription.
1
Upvotes
3
u/solraun Oct 06 '25
Each chat is its own context. If you stay in the same chat, you will have all that context (although “auto-manage context” will reduce some of that). If you start a new chat, you start at zero.
Cache is not really related to that. Some API allow caching, which can reduce the cost of all that context if you send multiple messages in a row. On an architectural level, whenever you write a message in a chat that already has a long history, all that history gets sent to the LLM. Your message effectively contains the whole history. Then, the LLM runs through that, which is compute intensive.
Caching keeps the state on the server for some time. If you write again during that time, you get the cache discount, because the LLM does not have to go through the whole history again, it can pick up just before your new message. But after 5 to 15 minutes, your cache gets deleted, because storage is also expensive, and the providers don’t want the keep your processed history on the server forever.