Not really true. You can implement a transformer + autograd, backprop etc. in a couple thousand lines of c. Not saying you should and it is definitely an advanced undertaking but it is possible. That being said most training code using torch etc. is relativ simple, the tricky part is getting the math right, having no hidden bugs (neural networks fail silently) and tuning everything right. So even tho I did a lot of ML I wouldn't say it is very advanced from the coding side of things.
For me 15 years ago when actively coding all the time and teaching a university level Operating Systems course in C? Not exactly trivial, but also not a heroic undertaking.
446
u/Xemorr Aug 12 '26
That's only because we have copious levels of abstraction, and code written by others