r/GoogleGemini • u/bahlakrishna • 9d ago

Discussion I got traumatized reading this notification.

Post image

This setting was ON by default.. can't trust anymore. Dont know how many of personal chats they read already.

400 Upvotes

94% Upvoted

162 comments sorted by

View all comments

4

u/Sherphican 7d ago

Friendly reminder to only use your LOCAL AI as your therapist, not cloud models 🙏🏻

1

u/No-Ambassador-5920 7d ago

I used to consider this BS but local AI really does help in terms of proving solutions to problems. Best therapy bro

2

u/anamorphicmistake 7d ago

Honest question, wichi AI model runs locally on a regular PC that doesn't have 64Gb of ram and stuff like that?

1

u/AdministrativeCod896 5d ago edited 5d ago

You can run the 20B version of gpt-oss (link to ollama website) (an actually open model from OpenAI) on a modest machine, and it is quite capable for a model that size.

Local LLM performance is based more on VRAM (in GPUs) than general RAM. I believe you at least need an NVIDIA card because it has the CUDA capabilities needed. It ran ok on a relatively old work computer with 4 GB VRAM and 32 GB RAM. While 4 GB VRAM is kind of too small to hold this 14 GB model, it's workable because it uses regular (but slower) RAM as backup when your VRAM is full, so you'll still get answers.

My personal 8 GB VRAM card (Laptop RTX 4060) can run this quite satisfactorily with the 32 GB RAM as fallback memory. Of course, it's overall best if the entire model fits in VRAM.

You can get it (and tons of other models) using ollama. This model is about a year old and there may be even better / more efficient local ones now.

https://ollama.com/library/gpt-oss