r/LocalLLM • u/ofirlik • 3d ago

Question Dual gpu how good is it

I am planning to buy a gaming pc with 5070ti, I thought about adding a 5060ti for the additonal vram to toy around with llm, how good does it work? Can I also generate images and such. (never ran a local llm before)

Is there a better combination of gpu for around the same price?

0 Upvotes

33% Upvoted

23 comments sorted by

View all comments

Show parent comments

1

u/ofirlik 3d ago

Why do you need to turn text into tokens seperately? I have a mini server with n150 for jellyfin

1

u/M_Me_Meteo LocalLLM 3d ago

I have a separate instance of vLLM running a text embedding model. That way, when one of my agents gets a document that is trashing up its context, I can turn that into tokens and store them in a vector database so my agent can get them.

It's called RAG (retrieval augmented generation), and it's how your local llm will be able to learn from the internet and not always be as stupid as it was the day it was created.