QR Factorization

General Gram-Schmidt procedure
The standard Gram-Schmidt procedure assumes that the input vectors $$a_1,\dots,a_k \in \mathbb{R}^n$$ are independent. If we remove the requirement of independence, what we have is the general Gram-Schmidt procedure. Clearly, the $$R$$ in the factorization $$A=QR$$ is not an upper triangular vector anymore, but it has the upper staircase form. Of course, we can always permute the columns via permutation matrices $$S, P$$ such that the $$\tilde{R}$$ in $$A=Q[\tilde{R}\,S]P$$ becomes an upper triangular matrix concatenated to the right with some other matrix.

Full QR Factorization
Assume that the $$QR$$ factorization of $$A \in \mathbb{R}^{m\times n}$$ is $$A=Q_1R_1$$. Note that the set columns of $$Q_1$$ have to be orthonormal, but $$Q_1$$ does not have to be an orthogonal matrix. Sometimes we want to "complete" it to an orthogonal matrix $$Q = [Q_1 \,\, Q_2]$$; this leads to the full QR factorization:

$$A = [Q_1 \,\, Q_2] \begin{bmatrix} R_1 \\ 0\end{bmatrix}$$.

This is pretty simple to achieve. Find any matrix $$\tilde{A}$$ such that $$[A\,\,\tilde{A}]$$ is full rank (e.g. $$\tilde{A} = I$$) and apply general Gram-Schmidt to $$[A\,\,\tilde{A}]$$. This operation would be called extending the original input set of vectors to an orthonormal basis.

$$Q_1$$ and $$Q_2$$ are very interesting: The ranges $$\mathcal{R} (Q_1)$$ and $$\mathcal R (Q_2)$$ are complementary subspaces, since
 * They are orthogonal, i.e. with an overloaded notation of $$\perp$$, we can write $$\mathcal R (Q_1) \perp \mathcal R (Q_2)$$
 * Their sum is $$\mathbb{R}^n$$, i.e. with the over-loaded notation of $$+$$, $$\mathcal R (Q_1) + \mathcal R(Q_2) = \mathbb{R}^n$$.

This can also be written as

$$\mathcal R (Q_2) = \mathcal R (Q_1)^\perp $$ and $$\mathcal R (Q_1) = \mathcal R(Q_2)^\perp$$.

Clearly, $$\mathcal R(Q_1) = \mathcal R(A)$$ and, less clearly, $$\mathcal R(Q_2) = \mathcal N (A^T)$$.

Connection to Least-squares and projection
The matrix $$QQ^T$$ is the projection matrix in the context of least-squares.