MAT4 (Ole Christensen) Review

The core concept of this class is approximation. The class provides the fundamental mathematical tools for approximation. Suppose that we have a function $$f \in L^2({\mathbb R})$$ where $$L^2(\mathbb R)$$ is a fairly generic space that covers a broad range of functions. The course uses the concept of denseness to show that any function in $$L^(\mathbb R)$$ can be approximated with a continuous function $$g$$ that has compact support (i.e., $$g \in C_c(\mathbb{R})$$). This statement requires the introduction of core concepts such as Banach spaces, Subset closure, Norm, Operator norm, Bounded linear operator etc.

The course starts from abstract and goes toward more concrete and applied concepts. Specifically, it starts with the more general topics of convergence in normed spaces, inner vector spaces, Banach spaces Hilbert spaces, bounded linear operators on Banach/Hilbert spaces etc. It then goes to specific Banach/Hilbert spaces such as $$L^2(\mathbb R), L^1(\mathbb R), C_c(\mathbb R), C_0(\mathbb R)$$. The concept of basis in finite and infinite spaces is introduced. The course defines specific critical Bounded linear operations on $$L^2(\mathbb R)$$ such as the Fourier transform or more basic operators such as translation operator, modulation operator and dilation operator. All those operators then lead to the very important Wavelet theory, which is of great utility for real life applications. The course introduces important inequalities that are used to prove the main results, namely Minkowsky and Holder inequalities and Cauchy-Schwartz inequality.

The definitions and theorems etc. referred to below are from the book.

Week 1

 * Introduces norm
 * Motivates usage of normed vector spoces
 * Proves negative triangle inequality
 * Definition of sequence convergence (Def. 2.1.5.)
 * Definition of subspace
 * Definition of supremum and "bounded above"
 * T.1.6.3: Continuous functions on a bounded interval attain their supremum

Week 2

 * Banach spaces
 * Cauchy sequences (Def. 3.1.1), and their importance; that is, we don't need to guess to what the sequence converges too (see Cauchy sequence).
 * Convergent sequence => Cauchy sequence (L3.1.3.). In many spaces, the opposite holds too; we call those Banach spaces.
 * $$C[a,b]$$ is a Banach space (T3.1.6).
 * $$\ell^1(\mathbb N)$$ and $$\ell^p(\mathbb N)$$ spaces.
 * Bounded linear operator
 * Operator Norm

Week 3

 * Inner product definition
 * Cauchy Schwarz inequality (T.4.1.2)
 * Defining norms from inner products (L.4.1.3)
 * Not all norms come from an inner product (T.4.1.4 helps us identify when a norm comes from inner product).
 * Hilbert spaces: The normed vector spaces whose norm originates from an inner product
 * $$\ell^2(\mathbb N)$$ is a Hilbert space (but not other $$\ell^p(\mathbb N)$$ spaces!)
 * Functional definition
 * Riesz' representation theorem. In a Hilbert space, any bounded functional can be shown to be the outcome of an inner product.

Week 4

 * Adjoint operator.
 * Properties of the adjoint operator
 * Self-joint and unitary functions (Def. 4.5.4.); those are generalizations of transpose and orthogonality.
 * Shauder basis (D.2.5.4)

Week 5

 * Definition $$L^1(\mathbb R):=\{f:\mathbb R \to \mathbb C : \int_{-\infty}^{\infty} |f(x)| dx < \infty \}$$.
 * How to establish membership to $$L^1(\mathbb R)$$.
 * That is, $$\int_{-\infty}^{\infty} |f(x)| dx$$ always makes sense; it can be finite or infinite but either makes sense. $$\int_{-\infty}^{\infty} f(x) dx$$ may or may not make sense ; for example  $$\int_{-\infty}^{\infty} \sin(x) dx$$ does not make sense -- it's not clear what it means.
 * L1.7.2: If $$\int_{-\infty}^{\infty} |f(x)| dx$$ is finite, then  $$\int_{-\infty}^{\infty} f(x) dx$$ is well-defined (\i.e., it makes sense).
 * Holder's Inequality (T.1.7.4)
 * Support of a function (D.5.1.1)
 * $$C_c(\mathbb R)$$ and $$C_0$$ are introduces
 * $$C_0(\mathbb R)$$ is a Banach space (L5.1.3)
 * $$C_c(\mathbb R)$$ is a subspace of $$C_0$$.
 * Counter-intuitively, $$C_c$$ is NOT a Banach space (L5.1.4).

Week 6

 * $$L^p(\mathbb R)$$ spaces (see special vector spaces).
 * Advantages and some properties of $$C_0(\mathbb R)$$ and $$C_c(\mathbb R)$$
 * Advantages and some properties of $$L^2(\mathbb R)$$ and $$L^1(\mathbb R)$$
 * Translation operator, dilation operator, modulation operator.
 * L.6.2.2: The operators above are well-defined, linear, bounded and unitary.

Week 7

 * Composition of operators. Starting from the outer operator is generally better.
 * L.6.2.3. On the compositions of translation, modulation and dilation operators.
 * Orthonormal system definition (D.4.3.1)
 * Orthonormal basis definition (D.4.7.1)
 * T.4.7.2: One of the central theorems of the book: Characterization of Orthonormal basis in 5 different ways.
 * $$L^2(-\pi, \pi)$$ definition
 * An orthonormal basis for $$L^2(-\pi, \pi)$$
 * Fourier series converges but not necessarily pointwise. I.e., convergence happens in terms of norm, which uses an integral, which ignores individual points.

Week 8

 * Beginning Approximation theory.
 * Definition of dense subsets.
 * L.3.2.4: spaces that contain sequences with finite non-zero elements are dense in $$\ell^p(\mathbb N)$$ (which are infinite spaces) for any $$p\in[1,\infty)$$
 * $$C_c(\mathbb R)$$ is dense in $$L^p(\mathbb R)$$.
 * Weierstrass' theorem: Any continuous function can be approximated arbitrarily well with polynomials (i.e., the set of polynomials is dense in the set of cont. functions).
 * Extension of bounded operators to allow operation on vectors who are not in their original domain.

Week 9

 * Fourier transform
 * Range of Fourier transform: i.e., which functions can be represented with a Fourier transform (see Fourier transform)?
 * Commutation of dilation and translation operators
 * Function of Fourier transform: it measures the frequencies in a given functin
 * Fourier transform is bounded
 * L.7.2.1: The norm of a function is equal to the norm of its Fourier transform
 * Calculation of Fourier transform in special cases:
 * When the function is even
 * When the function is odd
 * When the function is composed with translation operator
 * When the function is composed with modulation operator
 * When the function is composed with differentiation
 * Unitarity of Fourier transform (T.7.2.2)

Week 10

 * Paley-Wiener space: An important subspace of $$L^2(\mathbb R)$$
 * Shannon's sampling theorem (T.7.4.5):
 * Definition of convolution operator
 * Fourier convolution theorem: Convolution in the function's domain is equivalent to multiplication in Fourier domain.
 * Introduction to wavelets

Week 11 & Week 12

 * These are the weeks where the Wavelet theory is studied. It is shown how to consturct a wavelet system. This process is summarized in the article Wavelet.
 * The concept of Vanishing moments is introduced (D.8.3.2). This is good for compression (see T.8.3.3).
 * The theorem T.8.3.3 is a breakthrough moment as it theoretically guarantees that certain wavelet coefficients will vanish and guides the choice of good wavelet functions.
 * Applications of Wavelets:
 * FBI stores fingerprints with wavelets (a technique that had won a competition)
 * JPEG 2000, MP3 use wavelets
 * Noise reduction. It was apparently used to be able to discover the music that was recorded with a simple technique by Brahms; without noise reduction the scores were not comprehensible.
 * Splines and B-splines are introduced (D.10.1.2 for the latter)
 * Many interesting properties are of B-splines are listed in T.10.1.3 and C10.1.7
 * A great advantage of B-splines is that they lend themselves very well for computer-based computations:
 * They have compact support
 * All computations can be reduced to computer-friendly calculations of +/-/x/% as B-splines are piece-wise polynomials (C.10.1.7).