Fundamental Theorem of Linear Algebra

The fundamental theorem of has two components: (1) Conservation of dimensions and (2) Orthogonality of subspaces.

I. Conservation of dimension
For any $$A \in \mathbb{R}^{m\times n}$$, it holds that
 * $$\text{rank}(A)+\dim\mathcal N (A) = n$$

This can be interpreted as conservation of dimension. That is, $$\text{rank}(A)$$ is the dimension of set 'hit' by the mapping $$y=Ax$$; $$\dim\mathcal N(A)$$ is the dimension of the set of $$x$$ 'crushed' to zero by $$y=Ax$$. Those two dimensions must always add up to $$n$$.

II. Orthogonality of subspaces
For any $$A$$, the following two groups of subspaces are complementary subspaces:
 * $$R(A)$$ and $$N(A^T)$$
 * $$N(A)$$ and $$R(A^T)$$

Why is it Fundamental?
The fundamental theorem of linear algebra directly leads to the URV decomposition, which not only is of importance on its own (see here), but also directly leads to the SVD decomposition (see here), which is one of the most important results, if not the most important one, of linear algebra.

The Fundamental Theorem of Linear Algebra expresses the relationship between the four fundamental subspaces of a matrix, namely the ranges of the column and row space, and the nullspaces of the matrix as well as its transpose. There are a few questions to be asked here: why do we have 4 fundamental subspaces and not less or more? And why does the relationship between matrices is fundamental for linear algebra?

Answers to these questions are:
 * Subspaces are very rare kind of spaces, in the sense that one cannot induce arbitrarily many spaces from a matrix. That is, the solutions to $$Ax=b$$ with nonnull $$A$$ form a subspace if and only if $$b=0$$ (see exercises 4.1.1, 4.1.2 and flatness discussion in p163 of ). Thus, there are only the 4 afore-mentioned subspaces that can be induced from the system $$Ax=b$$.
 * Subspaces are called also linear spaces, because the range of every linear function $$f: \mathbb R^n \to \mathbb R^m$$ is a subspace of $$\mathbb R^m$$, and every subspace of $$\mathbb R^m$$ is the range of some linear function (see p169 ).
 * A function of a vector $$x$$ is linear if and only if there is a corresponding matrix $$A$$ such that $$f(x) = Ax$$ (see linear function).

The items above suggest that subspaces are intimately related to linear functions, which in turn are intimately related to matrices, which can induce only four subspaces. That's why the grandiose title of this theorem is perhaps justified.