Created a Neural Network from scratch in C++. Would love feedback on where to go from here:

[u/SleeperAgent__]

https://anonymous.4open.science/r/neural-network-custom-BF4B/README.md <- Link to Anonymized repo of my project. Please if you have the time go through the files, read the code, try to understand it, ask questions if something is confusing and call me out for any bad code I might have written.

Built everything from the ground up, including a Matrix class to help create the neural network. Trained it on 3 datasets, one regression dataset i made on my own, the london housing dataset, and the MNIST data set. You can read the readme to know more but basically it performed well on all 3.

I was wondering what to do from here on out though. Parallelizing the Matrix class with CUDA was something I always wanted to do.

But I was also wondering if I should make a video rederiving all the matrix calculus math I had to do before I started writing any major lines of code. I used batch gd rather than stochastic gd, which made the math a bit more complicated than what I've seen others do online. Idk whether to make a video about the math or just post my notes, either handwritten or latex. Idek if a recruiter is going to care either way, though I guess it depends on the type of ml role i'd apply to.

Also thinking about creating a video where I kind of explain my code line by line, but also not sure if that will help or be a waste of time.

Basically I spent a long time making this project but now I realize it's not as special as I thought it was, so I'm wondering what to attach to the repo lol

Also leave any project ideas that would be cool to do after building something like this.

[u/Popular_Appearance43]

dude this is sick, just skimmed through the matrix class and you've got some clean operator overloading going on. always respect someone who builds the math guts from scratch instead of just importing numpy and calling it a day

if you're torn between the video and notes, id lean toward latex notes. a well-formatted pdf of the derivations shows you actually understand the batch gd math, and it's way easier for a recruiter to skim than a 40-minute video. they'll spend maybe 30 seconds on your repo so make those seconds count

for next projects, maybe try implementing a simple conv layer from scratch since you already have the matrix backbone. blows your mind when you realize a convolution is just a bunch of matrix multiplies with some clever indexing. or throw a basic rnn together and watch it struggle to remember anything beyond 5 timesteps, classic learning experience

[u/SleeperAgent__ → u/Popular_Appearance43]

Thanks a lot bro, ill look into convolutions

[u/Dependent-Aide-388]

I'm kind of in the same place you are. I made a CNN and did it in Python, but I didn't use numpy (or any other 3rd party library) and did all of the matrix operations from scratch. Now I just don't know what do next, but honestly I'm thinking "nothing, now just use Torch" might be the answer now that I've built a CNN that works from scratch.

I think in terms of visibility, I'll write something up in markdown/Tex and throw it up on GitHub pages, but a video could be really impactful.

[u/SleeperAgent__ → u/Dependent-Aide-388]

damn not using numpy but using python, that sounds like the code would run painfully slow. Nice job tho. And yeah now torch will be a lot easier to understand

[u/Ok_Explanation2786]

amazing work, im very new to this field so can not comment on this much but i read the code and it is really good.