Eigenvectors and Eigenvalues

By definition, $$\lambda \in \mathbb{C}$$ is an eigenvalue of $$A \in \mathbb{C}^{n\times n}$$ if the characteristic polynomial becomes zero for $$\lambda$$, that is

$$\mathcal{X}(\lambda) = \det(\lambda I - A)=0$$.

This is equivalent to the following two things:

1) There exists a (right) eigenvector: There exists non-zero $$v \in \mathbb{C}^n$$ such that $$(\lambda I -A) v = 0$$, i.e.,

$$Av = \lambda v$$ any such $$v$$ is called an eigenvector of $$A$$ associated with eigenvalue $$\lambda$$.

2) There exists a left eigenvector: There exists non-zero $$w \in \mathbb{C}^N$$ s.t. $$w^T(\lambda I - A) = 0$$, i.e.,

$$w^T A = \lambda w^T$$. Any such $$w$$

Geometric interpretation
The eigenvectors of a matrix have a noteworthy geometrical property. Let $$\lambda$$ represwent any nonzero eigenvalue of $$A$$, let $$v$$ represent any eigenvector that corresponds to $$\lambda$$, and take the usual inner product. Then, the direction of the vector $$Av$$ is either the same as $$v$$ or its opposite, depending on whether $$\lambda$$ is positive or negative. The length of $$Av$$ is $$|\lambda|$$ times that of $$v$$.

Properties of Eigenvectors and Eigenvalues

 * If $$v$$ is an eigenvector of A with eigenvalue $$\lambda$$, then so is $$av$$, for any $$\alpha \in \mathbb{C}$$, $$a\neq 0$$
 * Even when $$A$$ is real, $$\lambda$$ and $$v$$ can be complex
 * When $$A$$ and $$\lambda$$ are real, we can always find a real eigenvector associated with $$\lambda$$: if $$Av = \lambda v$$ with $$A \in \mathbb{R}^{n\times n}$$, $$\lambda \in \mathbb{R}$$ and $$v \in \mathbb{C}^n$$, then $$A \Re v = \lambda \Re v$$ and $$A \Im v = \lambda \Im v$$. In other words, $$\Re v$$ and $$\Im v$$ are real eigenvectors, if they are non-zero (and at least one is).
 * Conjugate symmetry: if $$A$$ is real and $$v \in \mathbb{C}^n$$ is an eigenvector associated with $$\lambda \in \mathbb{C}$$, then $$\bar{v}$$ is an eigenvector associated with $$\bar{\lambda}$$. (Can be shown by taking conjugate of $$Av=\lambda v$$.)
 * When $$A$$ is symmetric, its eigenvalues are real.
 * The non-zero eigenvalues of $$A B$$ and $$B A$$ are the same even when the sizes of $$A$$ and $$B$$ are different. This way, it's very easy to compute, for example, the non-zero eigenvalue of an outer product $$u v^T$$ -- it's the number $$v^T u$$. This should also be the property [TBC] that is used to compute the PCA of matrices that are very large in one dimension [I need to check this]. However, the analogous property does not apply for the eigenvectors of $$AB$$ and $$BA$$.

What's the point of Eigenvalues or Eigenvectors?
Eigenvalues or eigenvectors are ubiquitous, and any list that will be compiled will be largely incomplete. But here are some purposes and uses of eigenvalues/eigenvectors...

In the context of LDSs

 * The poles of an autonomous LDS are the eigenvalues. They are also the roots of the characteristic polynomial of the system.
 * The conditions for the stability of a continuous-time LDS can be expressed via left or right eigenvectors.
 * The conditions for the stability of a discrete-time LDS can be expressed eigenvalues (see Modal Form).
 * Understanding behavior of LDSs: If the initial condition of an LDS is a right eigenvector, then we know that the system trajectory will remain within the line spanned by that eigenvector.
 * Understanding the behavior of an LDS #2: Independently of the initial conditions, the left eigenvectors give linear functions of the state that are simple (e.g., decaying exponentials) (see 11-11 in )

For quadratic forms (and positive definiteness etc.)

 * If a matrix is symmetric, then it's eigenvalues can be used to test if it's positive/negative (semi)definite (see Positive definite matrix)
 * For symmetric matrices, eigenvalues define upper and lower bounds on a quadratic form (see Quadratic form).

Matrix norm and gain of a matrix
TBC
 * Eigenvalues determine how big a matrix is (what's the largest gain of a matrix) in a given direction.
 * The ratio of the largest and smallest eigenvalues determine the eccentricity of an Ellipse.
 * Therefore, the largest eigenvalue is arguably the most standard matrix norm.