Matrix norm

Arguably, the most common norm for a matrix is the spectral norm, which is defined for a matrix $$A \in \mathbb{R}^{n\times m}$$ as

$$||A||:= \max_{x\neq 0}\frac{||Ax||}{||x||}$$.

Using the upper bound on a quadratic form of a symmetric matrix (see Quadratic form), it is easy to show that $$||A|| = \sqrt{\lambda_{\max}(A^TA)}$$, where $$\lambda_{\max}(A^T A)$$ is the largest eigenvalue of the (symmetric) matrix $$A^T A$$:

$$||A||{}^2 = \max_{x\neq 0}\frac{||Ax||{}^2}{||x||{}^2} = \max_{x\neq 0}\frac{x^T A^T A x}{||x||{}^2} = \lambda_{\max}(A^T A) $$.

The best way to visualize the concept of eigenvalues is ellipsoid.

Matrix norm as a generalization
Matrix norm is a generalization of concept of the norm for vectors. Technically, the norm $$||A||$$ of a vector is a different operator from the norm of a vector $$||v||$$. Fortunately, when the matrix $$A$$ is actually a (column) vector, the two definitions coincide. Let $$a \in \mathbb{R}^n$$, and let us compute the matrix norm on $$a$$:

$$||a|| = \sqrt{\lambda_{\max}(a^T a)} = \sqrt{a^T a}$$

Clearly, the rightmost quantity is the L2 (vector) norm of $$a$$. The first property listed below contains both matrix and vector norms.

Properties of Matrix Norm
Being a generalization of the vector norm, matrix norm possesses a lot of properties similar to that of the vector norm, namely :


 * for any $$x$$, $$||Ax|| \le ||A|| ||x||$$
 * scaling $$||aA|| = |a| ||A||$$
 * triangle inequality: $$||A+B||$$
 * Definiteness: $$||A|| = 0 \iff A = 0$$
 * norm of product: $$||AB|| \le ||A|| ||B||$$

Intuitive explanation
The matrix norm is the matrix norm tells us how "large" a matrix is. This is understood intuitively when matrix norm is thought as the (vector) norm $$||Au||$$ for the unit vector $$u$$ that maximizes the gain of a matrix.