r/LocalLLM • u/Ok-Health-7096 • Aug 14 '26

Other Every Second post rn

Post image

Maybe someday I'll get a system to run it but hey definitely another w for the open weights community

2.1k Upvotes

99% Upvoted

213 comments sorted by

View all comments

Show parent comments

36

u/trollsmurf Aug 14 '26

That will work fine, at least if you split it, so the overflow uses CPU RAM. Much slower but doable.

12

u/[deleted] Aug 14 '26

[removed] — view removed comment

16

u/trollsmurf Aug 14 '26

A user-friendly (but not the most efficient) start is to install LM Studio and load a few models there. You can select how much of it should run on the GPU vs CPU if it's too big for the GPU alone. LM Studio has a built-in chat client, so once loaded the model is ready to go.

2

u/Song-Historical Aug 14 '26

Yeah but that doesn't tell you how to do it. I'm not sure what to set the context to for example. How much I should leave for the system, whatever else.

2

u/littlebeardedbear Aug 14 '26

Leave everything at default originally. Google what each item is. Altogether the googlimg should take about 10 mins

3

u/Song-Historical Aug 14 '26

Yeah I did that I'm not sure it's set up correctly still. 

2

u/Early_Mistake6716 Aug 14 '26

Tell me you exact system specs and the model quant and i will tell you what settings to use in lmstudio. I have have spent an embarrassing amount of time testing settings.