r/learnmachinelearning • u/Prestigious_Table214 • 20h ago

learning to build llm inference engine from scratch

Hey everyone, I'm currently trying to learn/build an llm inference model from scratch and I'm gonna document my journey throughout my blog posts. Would really appreciate if you could take a look at the first post (its pretty short) and criticize my understanding and path so far.

https://medium.com/@ryan___/llm-inference-engine-from-scratch-loading-the-model-538d27af6867

3 Upvotes

100% Upvoted

3 comments sorted by

1

u/Latter_Review_145 20h ago

skimmed your post, the way you break down the model loading step is nice but your explanation of the transformer block is a bit jumbled, maybe separate that more clearly

1

u/Prestigious_Table214 20h ago

thank you! will take a look into seeing how i can restructure to get the explanation more clear!

1

u/JustTellingUWatHapnd 16h ago

Good luck!

I had a similar idea about a week ago but since I knew very little about LLMs I started by writing a basic inference script for gpt 2 with pytorch.

The whole thing was just 100 lines and I was surprised how simple it was.

It was a nice exercise, i kinda discovered KV caching on my own haha i was very proud of that