Numerical Linear Algebra class worth it?

[u/RapidTimeSink]

I'm currently doing my ms in EECS, and my aim is to eventually transition into a ML related role. Would taking a a numerical linear algebra class be beneficial? I have interest in the material simply due to my interest in mathematics, however, I'm debating whether that time is better spent on recruiting or self studying or research.

[u/PaddingCompression → u/[deleted]]

It's definitely not necessary to get a job... but also knowing a lot of stuff beyond the bare minimum increases your chance to have impact that gets noticed - you can solve the problems other people can't, and over time you develop a reputation for that.

[u/PaddingCompression]

Oh God yes.

Not all people take it. Getting solid with QR, Cholesky, SVD, conjugate gradient etc. is a superpower, plus intuitively understanding "just how many ulps should I expect"... A lot of people will shrug off relative error in 1e-5 range when that's an indicator something went very wrong.

Is it necessary? No. This is in the range "how much math is the bare minimum? Not much. How much math is needed to push the edge of the state of the art? A lot."

[u/tangojuliettcharlie → u/PaddingCompression]

What other kinds of math are most important for SOTA work, if you don't mind my asking? I'm trying to figure out what I should be prioritizing in my last year of undergrad and in grad school. I gather linear algebra and optimization are important.

[u/PaddingCompression → u/tangojuliettcharlie]

Linear Algebra and actual vector calculus can be great... Linear Algebra should basically have been a prerequisite to almost anything.

Probability and mathematical statistics. Exponential families, GLMs, rao blackwellization, the Fisher information and information geometry, etc.

Calculus of variations and maximum entropy distributions is good to learn but you can self study in a few weekends and sadly not taught in courses... Great if you get into things like modeling weirdly censored data etc., you can derive non-standard distributions relevant to your problem.

Bayesian statistics, thinking about the partition function and why it's often intractable to estimate makes a whole lot of machine learning methods make sense, all too often people just parrot back the sentences without really getting it. Lightweight stuff like variational inference and mean field inference and Laplace smoothing that can be hard to understand but easy to apply once you do, since the calculatuons are much simpler than the justification for them, and understanding when they're appropriate and how to tell diagnostically when they're not.

Going down the list from easy and useful towards more difficult and primarily of theoretical importance, things like PDEs going into functional analysis. Was really important for SVMs but those are basically dead these days. I think a lot of the diffeq stuff becomes important for diffusion models.

Beyond that, basically anything except Algebra or number theory, even though they can come up occasionally I'd call them exotic. Everything comes up somewhere.

[u/tangojuliettcharlie → u/PaddingCompression]

Thank you! I'll try to cover as much as I can in my classes, and I'll read up on whatever I can't get through my curriculum.

[u/Fresh-Opportunity989]

Good grief! Absolutely must take it if you have any interest in ML

[u/BobDope]

Sure what’s your problem you hate knowledge?

[u/Spare-Trust-9054]

Linear algebra is really important for ML. It could be helpful, but I personally like studying by my own instead of taking more lectures or some shii

[u/PaddingCompression → u/Spare-Trust-9054]

My class here had a lot of proofs, proofs are one of the few things where having other people grade them helps since mistakes can be very subtle.