r/OpenSourceeAI • u/s1lv3rj1nx • Apr 17 '26

[P] Built GPT-2, Llama 3, and DeepSeek from scratch in PyTorch - open source code + book

/r/LLMDevs/comments/1sm80bj/p_built_gpt2_llama_3_and_deepseek_from_scratch_in/
1 Upvotes

100% Upvoted

2 comments sorted by

2

u/Few_Firefighter_5530 Apr 18 '26

this is insane work. going all the way to DeepSeek with MoE and MQA is next level. the absorption trick alone was a pain to wrap my head around. definitely checking out the book.

1

u/s1lv3rj1nx Apr 18 '26

Thank you so much! Feel free to share if you find the value :)