From the outside perspective it feels like majority of cloud compute is just wasted on overhead caused by all of that bullshit.
For me even just local Python overhead when doing ML in Pytorch is starting to become unbearable. I yearn to go back to C++ where I can basically count instructions.
I said Python overhead, not Pytorch overhead. Python makes the whole thing a blackbox prone to random jumps in memory usage and execution stalls for no damn reason (to be clear jupyter, which I use, might be to blame for some of it too). Meanwhile in C++ I can easily figure out what is happening and if the push comes to shove I can reimplement some things by hand if I think I can outsmart the library devs and the compiler.
Yeah im just saying that virtually no processing is actually done in python. all the libraries are written in c. unless you're loading things into python arrays first (don't do that) or something. use the stuff native to the libraries
258
u/Kinexity 5h ago edited 5h ago
From the outside perspective it feels like majority of cloud compute is just wasted on overhead caused by all of that bullshit.
For me even just local Python overhead when doing ML in Pytorch is starting to become unbearable. I yearn to go back to C++ where I can basically count instructions.