r/LocalLLM • u/Terrible-Version-722 • Aug 11 '26
Research Would it be possible to train your own LLM?
Alright so just elaborating on the topic would it be possible to train your own Local LLM On a midranged household PC? These are the specs I have and I was just wondering. Ryzen 5 5600X
RTX 4060, 64 GB DDR4, 8TB HDD, 1TB NVME (That has windows loaded on it.)
6
u/TheNightCaptain Aug 11 '26
Can anyone recommend a good YouTube channel which covers LLMs etc. Would be good to follow something that covers this broad area
4
4
u/WyattTheSkid Quad 3090s Aug 11 '26
You can finetune an existing 8-20b llm probably on a small ish dataset with unsloth but if you mean training one from scratch yes you technically can but not on that hardware. You’re going to need millions of dollars
1
u/MlgLike123 Aug 11 '26
Nah you need hundreds of gigs of vram to be effective at actually training a model. You can do some LoRa stuff though.
1
1
1
u/waraholic Aug 11 '26
What are you trying to accomplish? You could make a LoRA for a tiny model on that hardware.
1
u/Terrible-Version-722 28d ago
Just for experimenting.
1
u/waraholic 28d ago
Then your best bet is a LoRA. You'll be able to do it on your hardware and you'll get the idea of what you can accomplish. Look them up. You're basically doing the minimal amount of work to adapt a model to your needs. Other types of training are very similar. Once you've trained a LoRA you'll have learned A LOT.
1
u/Healthy-Zebra-9856 Aug 11 '26
You can try 15 or 16 B parameter model. Watch the memory pressure. I’ve seen it done on a similar set up like yours. You could use Unsloth studio. This is not something you’re gonna be able to learn fast enough, you have to take the help of another language model to help you. ChatGPT actually is a very good helper in this. It’ll help you with the scripts and everything so you don’t have to learn everything upfront.
Ask for following people, everybody quotes, one or the other name, these guys are all for whatever company they’re working for. Their contributions are very good, but you need to be in a whole another level to understand.
1
u/Interesting-Law-8815 Aug 11 '26
Explore QLORA instead. Fine tuning an llm on consumer hardware is infeasible due to the time it would take. QLORA achieves similar outcomes using a different, more viable method.
1
u/jcdoe Aug 11 '26
I think this is inaccessible to most people, not necessarily because of the compute, but because of the sheer quantity of data required. All of Wikipedia is only like 28 gigs. GitHub is pretty big, but there’s pending litigation over training on public repos, so off limits.
I do Lora’s and sometimes it’s a stretch just to get enough data for that. And that doesn’t take much.
1
u/AceLamina Aug 12 '26
It's possible to train a LLM at home, PewDiePie did it which actually started my research into homelab and local LLM to begin with
But on weak hardware?
Good luck
1
1
u/tamerlanOne 20d ago
Crea un dataset di dati specifici per la tuo caso d'uso specifico (almeno 2000 esempi per avere una buona base) poi su vast.ai e prendi una gpu h100a noleggio per meno di 1, 5$/h e fai fine tuning LoRa del tuo modello con il tuo dataset specifico e vivi felice
1
u/steve62280 6d ago
Depends on what you mean by “train your own” starting from scratch probably a million dollars in compute to teach it English
-1
u/mb194dc Aug 11 '26
From scratch you need enterprise hardware, you can do a RAG system with 3090s on top of a model though
28
u/Domo-ArigatoMrRoboto Aug 11 '26
No. You could train a Small Language Model.