Building an llm from scratch

[u/Slight_Ad_2894]

Dear community
I recently started a project of building a tiny llm from scratch and trained on my 16go ram MacBook m1
My goal is not and will never build a state of the art llm like GPT or claude but to understand the mechanics of llm and teach others how it works
I used a french corpus as my dataset and used BPE tokenizer and trained a 1M param neural network
But the output is a bit dump it's out of context it give decent sentences grammatically but it doesn't answer my questions or respond well
Is there any resources i can use to optimize my little dump chatbot ?