## Saturday, April 18, 2009

### Eigendecomposition and Singular Value Decomposition

Eigenvalue and eigenvector are those satisfying the following eigenequation:

matrix(transformation) * eigenvector = eigenvalue * eigenvector

Thus, if we can find such a eigenvector and therefore a eigenvalue, their interpretations are: after being linearly transformed by the matrix, eigenvector still has the same direction. Eigenvalue can thus be considered some essential part of the matrix, or the characteristic value of the matrix. Eigenvector can be considered a tool to extract such essential part of the matrix.

A nice explanation can be founded here; see also Borg and Groenen (2005) Chapter 7.

Eigendecomposition: matrix A = QΛQ'
Thus, AQ = QΛQ'Q = QΛ, where Λ is a diagonal matrix of eigenvalues
Singular Value Decomposition: matrix A = PΦQ'

P is a matrix of left singular vectors, Φ is a diagonal matrix with singular values, Q is a matrix of right singular vectors. The naming choice of "singular" probably is similar to that of "eigen", because the expressions of the two decompositions are very similar and probably referring to the essential and unique quality of the matrix.