Pseudo-inverse

When talking about pseudo-inverse of a matrix $$A$$, we usually refer to a matrix denoted as $$A^\dagger$$ and is a specific generalized inverse of $$A$$. The form of this specific matrix depends on the type of matrix.

$$A^\dagger = (A^T A)^{-1} A^T$$. In this case, $$A^\dagger$$ is a left inverse.
 * If $$A$$ is skinny and full rank, we generally refer to:

$$A^\dagger = A^T (AA^T)^{-1}$$. In this case, $$A^\dagger$$ is a right inverse.
 * If $$A$$ is fat and full rank, then we refer to the pseudo-inverse:

$$A^\dagger = V \Sigma^{-1} U^T$$ where $$V, \Sigma, U$$ are obtained from SVD.
 * If $$A\neq 0 $$ (i.e., the most general condition), the typical pseudo-inverse is:

Equivalently, the pseudo-inverse in this most general condition can be represented as:

$$A^\dagger = A'(AA')^-A(A'A)^-A'.$$

The pseudo-inverse is also referred to as the Penrose-Moore or Moore-Penrose inverse.