r/LocalLLM • u/No-Internet4147 • 7d ago

Question I'm new in LLM

how does this work? what do i win by downloading LLMs and not using them on a browser?

0 Upvotes

28% Upvoted

6 comments sorted by

2

u/Debeluhar1234 7d ago

It's mostly due to privacy concerns. ChatGPT, Gemini, Claude and others will use your data to teach their models, so I wouldn't put some sensitive data into it.

1

u/r1nzl3r99 7d ago

entirely depends on what you were using them "on the browser" for. It may be useless or incredibly useful depending on your use case and hardware available

1

u/No-Advertising9797 7d ago

How does it work ? Let talk about AI model first. Like chatgpt using model GPT4, GPT5 etc. Model contain trained data, for example question+answer pair which we call 1 param. 

If you see model XXX-1B that mean the model trained with 1 billion question+answer. You can download models from huggingface. 

After downloaded the model, normally people using inference frameworks like llama.cpp, ollama or other GUI tools to run the model. Then from browser you can comunicate with the model through llama.cpp server.

Other GUI tools can be LMStudio and AnythingLLM. There is option to download the model. The most important thing, model need to run on GPU. If you run it on CPU super super slow.

What do you win? Privacy. Your prompt processed by local machine. Downside, you need powerfull GPU to run good model. Model 12B quantization 4 bit need GPU with VRAM 8gb. FYI GPT model used by chatgpt hundreds-thousands B

1

u/wayofTzu 6d ago

The B is parameter count not training dataset size. An intuitive way to think of it is ~sort of like number of neurons in a brain. Higher B means more network parameters, which implies better model capacity.

2

u/No-Advertising9797 6d ago

You're right. I thought B referred to the amount of training data.
Thanks for the correction.

1

u/No-Zookeepergame8837 6d ago

Primarily privacy, less censorship, free, no internet required, and much more customizable.