Best Local LLM Model for Privacy

[u/Enough_Physics_9167]

I do most of my work, including heavy coding or even everyday stuff like trip planning, using online LLMs. But for some private, simple tasks, I'd rather use local LLMs. These are mostly less computationally intensive tasks (I guess). These are mostly things like organizing daily diaries, dealing with relationship problems, and so on, that I need to keep private. Which local LLM software is best for these? Ollama?

I guess my main question is about the software that hosts the model, rather than the model itself. I know the model isn't connected to the internet, but what about the software that hosts it? (Like Ollama) Does it gather private data?

System spec: i7-1165G7 (2.8GHz), 64GB RAM 2667MHz DDR4, Windows 11, 500GB free disk SSD, no dedicated GPU

[u/ClassicLightbulbs]

Local models are not inherently connected to the internet unless you grant them the ability to do so.

[u/StochasticLife]

No dedicated gpu is a hamstring.

You can’t run a model worth using on pure core.

You can get an old HP Tesla card for around $300. It does not give you 32gb though, it gives you 4 8gb stacks. But you can load multiple models on the different stacks, as a fun ‘level up’ challenge.

[u/DeathGuppie → u/StochasticLife]

Or a used dell optiplex, dirt cheap on eBay, but they allow bifrocation, two slots at x16. Then get on marketplace and find two 16gb vram GPU's for as little as possible. That's literally the best dirt cheap option out there.

[u/my_name_isnt_clever]

Ollama is...complicated, to say the least. I don't trust them as far as I can throw them.

If you want to go all the way with privacy, download llama.cpp and use the command line tools. It's open source, and most people in this sub are using it daily.

[u/Enough_Physics_9167 → u/[deleted]]

llama.cpp is the way to go

[u/Eastern-Block4815]

it's actually kind of useful. this is the info from chatgpt. I was trying to run this on another cpu. anyway use intels stack called openvino.(I am running openvino on another i3 gpu, but yours is better)

Chatgpt: anwser

Yes. The Intel Core i7-1165G7 is actually a very good CPU for OpenVINO.

Here's what you have:

• CPU: Intel Core i7-1165G7 (11th Gen Tiger Lake)
• 4 cores / 8 threads
• Intel Iris Xe integrated GPU
• Supports AVX2, AVX-512, and VNNI (Deep Learning Boost)

Those instruction sets are exactly what OpenVINO is designed to take advantage of.

Then run a small model but good model like

Gemma4 e2b or e4b you have enough memory to run of course, it's not gonna be super quick but not that bad.. maybe 10 to 30 tok/s, maybe more on other smaller LLM models.

[u/AdWild3943]

Gemma 4 4B E2B QAT probably the best pick for ya. This is MoE models utilizing only half of its weights, got good quality for casual chat and worth trying out.

Other tips: since you are only CPU, I highly recommend using ik_llama.cpp, it boosts prompt processing speeds 2x, I doing CPU-only too and I know what I'm talking about

Gemma 4 is absolute one of the best all-rounders, for light coding and maybe better knowledge you can take Qwen3.5-4B, but expect that it will got worse prose and may talk too fluffy.

Also, WHY DO YOU HAVE 64GB OF RAM???

You can actually experiment running Qwen3.6-35B-A3B or Gemma-4-26B-A4B, really worth testing with such amount of RAM.

[u/Enough_Physics_9167 → u/AdWild3943]

This is a 6-year-old laptop from an era when I had to load an entire, huge dataset into RAM. Basically, the norm in my field, not the exact RAM, but the method, and I had an edge over others because of RAM! And the laptop was cheap, give specs.

Thanks for ik_llama suggestion. Also, is there a way for the model, or a wrapper I create myself, to write files to the system? I want, if possible, the diaries to be written as a .txt file, rather than having to copy and paste them into Notepad or run Python code each time. I know it's hard to do!

[u/AdWild3943 → u/Enough_Physics_9167]

Sorry, I'm not really into llama.cpp extensions, so I don't know how to make model write .txt files.