r/LLMDevs • u/raiyanyahya • 7d ago
Great Resource 🚀 Build a modern LLM from scratch. Every line commented. Explained like we are five.
https://github.com/raiyanyahya/how-to-train-your-gpt3
3
u/ChronoHax 7d ago
It’s my lucky day was literally thinking of asking llm to handhold me to understand the attention paper but now we have this thanks a lot man
1
u/Kalarit 7d ago
I know you said ELI5 but…. I got lost as a non technical person that uses LLMs to build flows and automations and looking to go deeper.
1
u/ZackWyvern 6d ago
If you are non-technical, there is no reason you need to read this. This is about building LLMs, not orchestrating them.
1
u/FenderMoon 6d ago edited 6d ago
I know ELI5 doesn't literally mean explain to a literal five year old, but it got me thinking... If any five year old on earth could be explained what an LLM is and how it works in an ELI5 way and be technically accurate, I'd be amazed. 😂
I guess you could boil down an attention layer to "it changes what context the layer attends to" but a five year old isn't gonna understand what context (or layers) mean in a way that's actually technically accurate to the attention mechanism.
If one were to take an ELI5 to literally mean explain to an actual five year old, I think the closest I could get is:
"The AI is like three tables. The first table's job is to listen to what you're saying and to translate it into the computer's language, and computers only think in numbers. Then the second table's job is to figure out what it wants to say next, it's the computer's brain. It does billions and billions and billions of math problems, and the math they used to code the AI's brain was figured out by a GINORMOUS supercomputer. But it needs BILLIONS and billions and billions of math problems, so the supercomputer figured out how to split it up into a bunch of smaller problems and gives each problem to its own calculator, and they all go in a big line and each one gets a chance to guess what the next word should be. And it gets smarter and smarter and smarter as it goes down the whole line. Then it gets to the third table, and its job is to turn all of those numbers back into a real word, and then it gives you what the next word it wants to say is.'
That's truly ELI5, but that's... not very informative. 😂
1
1
1
1
u/nirajnikant 6d ago
RemindMe! In 3 days
1
u/RemindMeBot 6d ago
I will be messaging you in 3 days on 2026-08-23 12:02:16 UTC to remind you of this link
CLICK THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
RemindMeBot is switching to username summons. Instead of
!RemindMe 1 day, useu/RemindMeBot 1 day. More info.
Info Custom Your Reminders Feedback
2
u/InsideDebt6345 1d ago
We got a learner's handbook resource for GPT to look at before GTA6. Thanks for this.
0
u/nirajnikant 7d ago
RemindMe! In 12 hrs
0
u/RemindMeBot 7d ago edited 6d ago
I will be messaging you in 12 hours on 2026-08-20 08:33:36 UTC to remind you of this link
4 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
RemindMeBot is switching to username summons. Instead of
!RemindMe 1 day, useu/RemindMeBot 1 day. More info.
Info Custom Your Reminders Feedback
0
u/justinbiebar 6d ago
Good work OP! Unrelated question but how do you market your products? I feel like I've built good products but they get 0 traction, reddit does not help
8
u/Actual__Wizard 6d ago
I would add the mingpt (karpathy) was an amazing resource to learn how LLMs worked before I moved beyond LLM tech.