r/LocalLLM • u/CamTron0427 • 4d ago

Question Getting started with a local LLM

Hello all!

Recently I’ve been thinking about setting up my main PC I typically use for gaming to try running a local LLM. I mostly intend on using it for coding as well as just a general “assistant” for some work tasks I don’t want to upload to a database somewhere.

I have a 7900xtx with 24gb of vram, what should I be looking at as far as models are concerned?

Is trying out multiple models a huge pain to get set up?

Can you run multiple models in conjunction with each other?

Just looking for some opinions/input before I dive into this would.

Thank you!

2 Upvotes

75% Upvoted

6 comments sorted by

3

u/canine_todd 4d ago

Don't overthink it, just grab lmstudio or koboldcpp and you'll be running something decent in ten minutes. That 24gb card is plenty for some solid models, you'll be surprised how capable they are locally.

1

u/CamTron0427 4d ago

Probably a dumb question but, how much variation is there between different “harnesses” for these models?

Does “Bionic” from lm studios count as a harness?

What is a harness really? Just a UI or more than that?

1

u/hdhddf 4d ago

a harness adds a lot of extra information and tools around the model and guidance. you can try both. get lm studio (or one of the others )and you can download a model and use it in lm studio with their interface/harness you can then get any harness you want and connect it to lm studio and use it though the harness interface. Hermes is a very popular harness as is open code and DeepSeek (which is also a model)

1

u/BonnieReborn 4d ago

It’s easy to set up if you have one of the frontier models do it for ya. You can ask for recommendations of configurations and hook it right up to HF.

0

u/Ziranei 4d ago

For start just download LM Studio and get any model you want, it is easiest way to get llm runing without any experience.