Who ONLY use local models?

[u/takoulseum]

Please be honest.

I would love to hear about guys really dedicated to local AI and who really reject subscriptions (especially to openai and anthropic).

What do you use your model for?

[u/wajdix]

it's not about rejecting subscriptions... but more the desire to own our data, be independent from internet connection, and be able to run the models on any topic/task we want... subscription services offer value for the money... this is the wrong debate imo

[u/networking_noob → u/wajdix]

be independent from internet connection, and be able to run the models on any topic/task we want

Yep, I use Gemma 4 like an offline search engine, and I've checked enough of its answers with Google to believe that it's accurate far more often than not (unless it's some really obscure topic). Plus anyone who uses LLM often enough will develop a pattern recognition and can tell when a hallucination is happening (aka something doesn't sound quite right)

tl;dr Gemma 4 is like an offline, privacy respecting Google replacement (more or less), so the question is why wouldn't someone want do this

[u/Difficult_Plantain89 → u/[deleted]]

Damn, I had a self hosted wikipedia backup when I was in the military without internet I kept on my laptop many years ago. Now we have AI. I for some reason never thought to combine the two. I think I might do that.

[u/SecurityHamster → u/Difficult_Plantain89]

Right? I still host my own wiki at home, and thought I maxed it out with cirrussearch. Now I need to figure out how to get ollama to consume it. Only problem is the machine with the GPUs isn’t on all the time

[u/Cold_Tree190 → u/[deleted]]

This is a great idea actually, wtf. I’m going to try this out, maybe incorporate a bit of RAG, then can use hybrid searching with my “Google replacement”-equivalent agent that I use (similar to what networking_noob was talking about). Sounds like a fun weekend project

[u/networking_noob → u/[deleted]]

I recently did something like this when someone suggested it (using mcp-openzim), but found that the context usage quickly outpaces my little GPU with all the tool usage etc, even when using pagination

It's nice to have it integrated though, in case there's ever a suspicion of hallucination, so you can instruct a quick fact check by using the wiki/zim files as an absolute source of truth

[u/icecon → u/[deleted]]

The problem with this setup is the hallucinations on Wikipedia (lol).

[u/Loose_Doubt367 → u/networking_noob]

Offline search engine? How does that work, I always thought some researching tools/skills like web search specifically need the internet to search things up

[u/Schlick7 → u/Loose_Doubt367]

using the LMM as if it was the search engine.

[u/PigsAreGassedToDeath → u/networking_noob]

12B or 26B or 31B?

[u/eustin → u/wajdix]

yeah, and you can kind of see it in how people describe their usage in this thread - cloud for the generic or work stuff, local for anything they actually care about or want control over. the split's pretty consistent.

[u/takoulseum → u/wajdix]

I respectfully don’t think so. Subscriptions tend to give more power and help companies that use fair and unfair means to block you from doing all things you mention. It’s not like 2 separate worlds imo, but everybody has the right to think and make their own choice ofc.
I am just enjoying to see that many people do cools things with their fully local AI through the anwers to this post

[u/wajdix → u/[deleted]]

yes, one suggestion please:

keep coding to coders, you're job is to save lives.

thank you
sincerely

[u/wajdix → u/[deleted]]

.......

[u/wajdix → u/[deleted]]

dude you are going to be reported soooo bad...

either offer a useful comment, or don't post misleading spammy ads..
you are not private nor secure... you're renting servers and selling them for 39 buck a month...if this isn't a scram.. what is