Matrix multiplication

Matrix-to-Matrix multiplication
There are various ways to interpret the matrix multiplication, each of which can be useful in different contexts. For $$A \in \mathbb{R}^{m\times n}$$ and $$B \in \mathbb{R}^{n\times p}$$, the matrix product $$C=AB$$ can be interpreted as (see also page #98 in ):


 * $$C = [c_1, \ldots, c_p] = AB = [Ab_1, \ldots, Ab_p]$$.

It can also be interpreted the $$i$$th row of $$A$$ acting (on left) on $$B$$: C = \begin{bmatrix}\tilde{c}^T_1\\ \vdots \\ \tilde{c}^T_m\end{bmatrix} = AB = \begin{bmatrix}\tilde{a}^T_1 B\\ \vdots \\ \tilde{a}^T_m B\end{bmatrix} $$

It can also be interpreted in terms of inner product, i.e., the $$ij$$th element of product, $$c_{ij}$$, can be written as
 * $$c_{ij} = \tilde{a}_i^T b_j$$

Another interesting interpretation is in terms of summation of rank-one matrices obtained by multiplying the columns of $$A$$ with rows of $$B$$ (see formula 2.9 in ):
 * $$C = \sum_{i=1}^n a_i \tilde{b}_i^T$$

Matrix-vector multiplication
The vector $$y=Ax$$ can be interpreted as having rows obtained via inner product:
 * $$y=Ax=\begin{bmatrix}\tilde{a}^T_1 x\\ \vdots \\ \tilde{a}^T_m x\end{bmatrix}$$

The $$x$$'s that satisfy each of those inner products, $$\tilde{a}_i^T x$$, can be interpreted as a hyperplane in $$\mathbb{R}^n$$ with a normal vector $$\tilde{a}_i^T$$ and an offset of $$y_i = \tilde{a}^T_i x = \alpha$$. (See EE263 lecture notes of Stephen Boyd, slide 2-32, for a very helpful visual representation).

Why are these different interpretations important?
Those interpretations allow us to have a better intuitive understanding of matrix-to-matrix multiplication. At certain occasions, it may be easier to prove things using the column or row interpretation of matrix multiplication. Also, a very practical consequence is this: if we are going to use only one row or column of $$C$$, then we don't have to multiply the entire matrices $$A$$ and $$B$$ -- we may just multiply the corresponding row of $$A$$ with $$B$$ (or column of $$B$$ with $$A$$).