Linearization

This page considers linearization via first-order Taylor expansion. While this is a rather simple linearization, it is probably also the most common one.

Assuming $$f:\mathbb{R}^n\to \mathbb{R}^m$$ is differentiable, the first-order linear approximation of $$f(x)$$ around $$x_0$$ is :

$$f(x)\approx f(x_0) + Df(x_0)(x-x_0) $$

where $$Df(x_0)_{ij} = \left.\frac{\partial f_i}{\partial x_j}\right|_{x_0}$$ is the derivative (Jacobian) matrix. More technically, approximation means that:

$$x \text{ near } x_0 \implies f(x) \text{ very near } f(x_0) + Df(x_0)(x-x_0). $$ The term "very near" means that the error on the RHS above is at the order of square of the error on the LHS.

A mnemonic to remember the dimension of $$Df(x_0)_{ij}$$ is to realize that for the dimension of the right hand side of the equation above to be consistent, $$Df(x_0)(x-x_0)$$ needs to be the same size as $$f(x)$$, i.e. an $$Df(x_0)(x-x_0) \in \mathbb{R}^m$$. Since $$(x-x_0) \in \mathbb{R}^n$$, $$Df(x_0)$$ must be of size  $$m\times n$$.