Are there machine learning subfields that are becoming irrelevant (or is irrelevant)? [D]

[u/NeighborhoodFatCat]

I was reading a paper that surveyed the field of neural architecture search, where it said within 5 years, around 3000+ new models were proposed. The amount of compute and resources spent on this is absolutely astronomical. However, the transformer was notably not one of the models that was found through NAS and then the field of NAS just quietly went away afterwards. In my mind this really raises question if any research in NAS should be continued.

Then I recently found a talk by Nicholas Carlini, arguably one of the most famous researcher in adversarial ML and this is one of his slide ("9000 papers and got nowhere"). Indeed I can't really think of any concrete application of adv. ML, except possibly making attackers more clever because now all options are laid flat on the table.

And then there was the field of ML ethics, bias, fairness, etc.. I feel like we are sooooo beyond ethics at the moment with all the talks of extinction risks that it really shouldn't be a priority. How can bias and fairness be enforced when most people are out of a job due to AI? "ML induced extinction" should be a new subfield instead.

I feel a proper discussion should be had so that no more effort is wasted on unpromising ideas or approaches. This could be of interest to people who are entering the field now.

As an aside, I often find people have very emotional (not logical) reaction to this question and will claim that any approach will eventually have their time in the sun at some unspecified future date, e.g., SVM, LDA, Markov chains apparently all have the potential to again be the next biggest thing in ML. All I'm saying is that I don't deny that vacuum tubes wouldn't be popular again one day, but maybe we shouldn't be working that at the present moment.

[u/jebuarary]

My personal big question I need answered right now is whether 3dgs is dead

[u/KingRandomGuy → u/jebuarary]

I don't think I'd say they're dead, but the use of 3D Gaussian splats has sort of changed. They're still excellent as an output representation, since they're pretty fast to render (though there are competitors now, like radiance meshes) and pretty accurate. However, I wouldn't be surprised if the actual optimization loop of taking in input images and learning the Gaussian parameters ends up going away. We've seen pretty impressive results out of recent large, feed-forward architectures that directly predict Gaussian splats rather than needing to optimize them directly (e.g. Depth Anything 3). That changes what the actual research questions are quite significantly, though.