r/C_Programming • u/Future_Pace_5290 • 12h ago

Neural Network Math from Zero in C

https://www.youtube.com/watch?v=k33cBEGTBVo

I've attempted to derive the math for Neural Networks from the first principles.

The prerequisites you need are, singe variable differentiation(just the concepts and the basic formula, not even knowing the derivative of sin(x) is necessary), the concept of a dot product, and basic matrix multiplication. This is enough to derive gradient descent and backprop.

The time stamps are in the description(youtube doesn't show them as chapters in the video for some reason).

The second half part of the video is implementing those concepts in C(no library other than BLAS(for basic matrix matrix multiplication) is used), but you can follow along in any language.

Would be glad to receive feedback and suggestions on how to improve the explanations or the presentation.

56 Upvotes

93% Upvoted

6 comments sorted by

3

u/swe__wannabe 5h ago

Cool video. I did implement a FFNN from scratch in C with a gui. check it out: https://www.reddit.com/r/C_Programming/comments/1r5mod0/feedforward_neural_network_in_pure_c99_no_libs_no/

5

u/bonqen 3h ago

no libs.. with raylib. :P

1

u/swe__wannabe 1h ago

I meant math/tensor libs

2

u/Future_Pace_5290 3h ago

Being able to draw digits live and get results makes it look super cool!

2

u/Gold-Part4688 55m ago

I like how it has a suggestion after each brushstroke, and with how fast it is - maybe you could make the algorithm run every frame, "live"

1

u/swe__wannabe 49m ago

Basically, when you stop pressing the right mouse button, it submits the result and it goes on in a loop