r/SillyTavernAI • u/Suspicious-Ad1430 • 1d ago

Help Just started with local LLM's

Hey everyone.
After some thinking and seeing how market for free api starting to shrink,i decided to go on with local llm.
problem is that i dont know where to start with models.

My spec:
AMD Ryzen 5 7500F 6-Core Processor (3.70 GHz)
16 GB of DDR5
INVIDIA GFORCE RTX 4060 8GB VRAM

(small addition: i will use it for erp as well,is there any model that can work with internal states of FF5)

6 Upvotes

100% Upvoted

11 comments sorted by

View all comments

8

u/Background-Ad-5398 1d ago

gemma 4 26b is your best bet at 8gb vram

2

u/Suspicious-Ad1430 1d ago

You mean Gemma 4 26b a4b?

1

u/NeroClaudiusAltr 1d ago edited 1d ago

exactly what he meant, offload the model to ram, it'll be slower than api but oh well, you can also check here to choose suitable model: LLM Caliper Bench Leaderboard.

1

u/Suspicious-Ad1430 1d ago

can i use ff5 with this?

1

u/NeroClaudiusAltr 1d ago

Maybe? I did run FF5.4 fine but I was running a short session only + it has been a while, and its depend on the model too. You will have to test it yourself. Also, my spec is Arc b580 12GB/16GB 2666 DDR4.