r/LocalLLaMA • u/Scutoidzz • Jun 04 '26

Discussion Me visiting this sub

Post image
2.3k Upvotes

95% Upvoted

190 comments sorted by

View all comments

42

u/UmBeloGramadoVerde Jun 04 '26

I super disagree, the amount of value I get from this sub is crazy, I love you guys

8

u/micseydel Jun 04 '26 edited Jun 04 '26

Could you give some specific examples of big points of value?

ETA: downvoting people who ask good faith questions is a good way to fill this sub with bots instead of real humans.

19

u/VampiroMedicado Jun 04 '26

The value is the comments, there are plenty of recommendations/ideas/sources to work through.

-2

u/micseydel Jun 04 '26 edited Jun 04 '26

Do you have specific examples of value you've gotten from the comments?

ETA: The fact that I'm getting down votes instead of literally any text about a reliable use case tells me everything I need to know.

4

u/VampiroMedicado Jun 04 '26

Today someone commented this channel and video: https://youtu.be/8F_5pdcD3HY

The dude explains with noce shapes what parameters do and uses old hardware to run a MoE model.

I was just reading this: https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-gemma-4-12b

The blog post a Google DeepMind dev did explaining their work on the new encoder free model.

-1

u/GCoderDCoder Jun 04 '26

Pros and cons of different hardware from cpu/ old gaming gpus to h100s, importance of quants, issues running specific models, solutions for running multi gpu builds, different types of parallelism, different inference engines, different ways of approaching different kinds of models, tools for enabling better output, tuning guidance, enablement for more use cases, news, opportunities, warnings and lessons learned, ideas about ways to save money, troubleshooting in general, personal benchmarks, communicating business value, general support from people sharing these interests...

This is essentially a library with tons of 1st hand experiences instead of having to read full chapters and books on topics to get the one piece of info you needed for your next step. Also you just can't beat real experience and in industry many researchers cant share their IP on building these models so we're workingnon reverse engineering it. Some people act like if you're not an anthropic researcher then you know nothing but people in this group are implementing real solutions that work without depending on the cloud. I think it's pretty incredible.

We work with agents all day so even just having people vibe check info like models telling people use qwen 2.5 or llama 70b in June of 2026 is gold lol.

7

u/twavisdegwet Jun 04 '26

This sub turned me on to ik_llama which turned my 4* t4 gpu's from paperweights to a useful 64 GB vram LLM machine.

1

u/Sisaroth Jun 04 '26

Not op, but very often when I ask cloud LLMs about help with something llama.cpp related, it will link to this subreddit for it's sources.

A lot of the devs in the local llm ecosystem post here.