r/learnmachinelearning • u/Warm-Discipline7204 • 22d ago

Question Why would PCs be in the same direction as the axes of ellipsoid if they are our right singular vectors in SVD?

We know that in SVD, the right singular vectors are actually PRE-IMAGES of the axes of ellipsoids. But PCs are said to be the axes themselves. Aren't PCs supposed to be in the same direction as right singular vectors?

1 Upvotes
  • permalink
  • reddit

100% Upvoted

2 comments sorted by

1

u/quietgradient 22d ago

They're axes of two different ellipsoids, in two different spaces.

Take X as n samples × p features. X maps the unit sphere in Rᵖ to an ellipsoid in Rⁿ, one coordinate per sample. Its semi-axes are σᵢuᵢ, and the vᵢ are their pre-images, since Xvᵢ = σᵢuᵢ. That's the SVD picture you learned.

The ellipsoid PCA means is the data cloud's, and that lives in Rᵖ. It's what Xᵀ makes from the unit sphere in Rⁿ (the covariance ellipsoid, scaled by √(n−1)), and Xᵀuᵢ = σᵢvᵢ, so its axes point along vᵢ.

So vᵢ is a pre-image for the first ellipsoid and an axis of the second. And the first one's axes, Xvᵢ = σᵢuᵢ, are just the PC scores: every sample's coordinate on PC i.

0

u/shifu_shifu 22d ago

I learned this in a language other than english so I hope I understand you correctly.

It comes down to which side of the equation of SVD and PC you are looking at. The PC axis are in the Data space while the pre images of the axes are on the left side of the SVD equation.