Least-norm solution

When a linear system $$y=Ax$$ has more than one solution, we may be interested in getting the solution that has the least L2 norm, $$||\cdot||$$. Clearly, this can happen only if $$A\in \mathbb{R}^{m\times n}$$ is strictly fat. In other words, $$x$$ is underspecified -- many choices of $$x$$ lead to the same $$y$$. If we assume that the $$A$$ has full rank, according to the fundamental theorem of linear algebra, $$\text{dim} \mathcal N (A) = n-m$$.

The least-norm solution $$x_{ln}$$ is then :

$$x_{ln} = A^T (AA^T)^{-1}y$$

This can be shown by first showing that $$(x-x_{ln}) \perp x_{ln}$$, and using this to expand $$||x||{}^2$$ as $$||x||{}^2 = ||x_{ln} + x - x_{ln}||{}^2 = ||x_{ln}||{}^2 + ||x-x_{ln}||{}^2 \ge ||x_{ln}||{}^2$$

Geomteric Interprtation
The orthogonality condition leads to the fact that $$x_{ln} \perp \mathcal N(A)$$. The projection interpretation of the least-norm solution is that $$x_{ln}$$ is the projection of $$0$$ on the solution set $${x | Ax= y}$$.