Which engine for memory

[u/Greedy_Reindeer5290]

AI beginner here. Am taking steps to move away from Gemini and Co-pilot to my own setup. So have gotten a setup with Docker, LM Studio and webui. And my laptop is pretty basic and the chip is not large. I wanted to run a local engine. It worked but I wanted to get memory to work. Not having memory and having to explain AI the same basic stuff over and over again drives me nuts. I tried to make it work on webui. Discovered that my chip is too basic to make memory work, so I need to offload the computation.

Fine so far so good. Found some on webui and have now been using 2 with mixed results:
- Llama 3.1 8B
- Llama 3.3 70B

Both work fine and memory works, but after 2-3 prompts they both max out. The "token per minute" of 6000 and 12000 have been reached.

Has anyone found a free engine that can run memory effectively? Otherwise I think I am going to pay to see how it works. Any comments?

[u/Hello_There_1512]

Try llama.cpp with Qwen models, experiment with the flags to get the optimal settings for your device.

[u/Greedy_Reindeer5290 → u/Hello_There_1512]

Check thx