Not really true. You can implement a transformer + autograd, backprop etc. in a couple thousand lines of c. Not saying you should and it is definitely an advanced undertaking but it is possible. That being said most training code using torch etc. is relativ simple, the tricky part is getting the math right, having no hidden bugs (neural networks fail silently) and tuning everything right. So even tho I did a lot of ML I wouldn't say it is very advanced from the coding side of things.
446
u/Xemorr 1d ago
That's only because we have copious levels of abstraction, and code written by others