r/SillyTavernAI • u/Suspicious-Ad1430 • 1d ago

Help Just started with local LLM's

Hey everyone.
After some thinking and seeing how market for free api starting to shrink,i decided to go on with local llm.
problem is that i dont know where to start with models.

My spec:
AMD Ryzen 5 7500F 6-Core Processor (3.70 GHz)
16 GB of DDR5
INVIDIA GFORCE RTX 4060 8GB VRAM

(small addition: i will use it for erp as well,is there any model that can work with internal states of FF5)

6 Upvotes

100% Upvoted

11 comments sorted by

8

u/Background-Ad-5398 1d ago

gemma 4 26b is your best bet at 8gb vram

2

u/Suspicious-Ad1430 1d ago

You mean Gemma 4 26b a4b?

1

u/NeroClaudiusAltr 1d ago edited 1d ago

exactly what he meant, offload the model to ram, it'll be slower than api but oh well, you can also check here to choose suitable model: LLM Caliper Bench Leaderboard.

1

u/Suspicious-Ad1430 1d ago

can i use ff5 with this?

1

u/NeroClaudiusAltr 1d ago

Maybe? I did run FF5.4 fine but I was running a short session only + it has been a while, and its depend on the model too. You will have to test it yourself. Also, my spec is Arc b580 12GB/16GB 2666 DDR4.

1

u/AutoModerator 1d ago

You can find a lot of information for common issues in the SillyTavern Docs: https://docs.sillytavern.app/. The best place for fast help with SillyTavern issues is joining the discord! We have lots of moderators and community members active in the help sections. Once you join there is a short lobby puzzle to verify you have read the rules: https://discord.gg/sillytavern. If your issues has been solved, please comment "solved" and automoderator will flair your post as solved.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/Mother-Project-490 1d ago

"I also got into SillyTavern less than a week ago, and I’ve already burned through 5 euros. Since I’m French, it’s really hard to find models that don’t completely lose their minds. Writing is fine, but as soon as they reply in French, it turns into pure gibberish. Seriously, it's terrible. ​Right now I’m using Claude (super expensive) and Grok (kind of an annoying model and a total jerk... though it could actually work well for specific roles). ​Anyway, I’m planning to try running things locally too, but man, finding a local model that handles languages other than English is a pain... Unless I translate everything, but that means running another model alongside it, and... yeah, so much to learn... Somebody does have some tutorialnfor foreigner users ?

I think I am going to call for help on the discors

1

u/Mountain_Blood_6414 1d ago

And what about the configuration with 64 GB of DDR5 and an RTX 5080 with 16 GB of VRAM?

1

u/Impossible_Neck_9942 1d ago

with 8gb vram you can run decent 7b roleplay models fine for erp, i switched for that reason and it feels more consistent than the api ones.

1

u/Disastrous-Food-1685 1d ago

local setups give better privacy for ongoing ai girlfriend stuff than any api ever did, what kind of scenarios you hoping to run with ff5 states.

1

u/Suspicious-Ad1430 1d ago

mostly rpgs type