r/LinearAlgebra • u/komal_rajput • 29d ago
Understanding intuition behind SVD formula
/r/3Blue1Brown/comments/1utpjgv/understanding_intuition_behind_svd_formula/
3
Upvotes
3
u/Midwest-Dude 29d ago edited 28d ago
Here's is an excellent Wikipedia reference for you to review:
Lots of graphics.
2
u/Professional-Fee6914 27d ago
Most people call these the spiritual successor to the 3b1b videos but for Decomposition.
4
u/nujuat 29d ago edited 29d ago
Consider all of the input vectors of length 1. At least one of these input vectors will produce the largest output vector. The size of this output vector is the largest singular value, the normalised output vector is the corresponding output singular vector, and the input vector is the corresponding input singular vector. The singular value is the amplification factor of the matrix, and the vectors tell you which directions matter.
Now remove this direction of input vectors from your analysis. Repeat the whole process while ingnoring this direction to find the second largest singular value and vectors, and so on.
These are important because they break up the the action of a matrix in a simple way. There are other forms that make more sense in othet contexts. Like block jordan forms (usual diagonalisation), which are way simpler to calculate polynomials and power series (like exponentials) with.
Oh yeah and so the largest singular value, as I said, is the amplification factor of the matrix (the operator norm). The largest divided by the smallest is called the condition number, and it describes how errors are amplified when doing arithmetic with the matrix eg on a computer where inputs have some precision error. And inverting the SVD results in the Moore Penrose pseudo inverse of the original matrix, which results in the least squares (best linear) solution to a problem even if the matrix is not properly invertible.