$\renewcommand{\R}{\mathbb{R}}$
$\renewcommand{\1}{\mathbb{1}}$

# Lecture 3: Separability of $L^2$, Projections, Weierstrass Theorem
###### tags: `224a`
In the last lecture we showed that a Hilbert space is separable if and only if it has a countable basis (indeed, these two terms are often used interchangeably). We will now show that the examples we looked at are separable.

It is easy to see that $\ell^2$ is separable: the set $\{e_n\}_{n=1}^\infty$ consisting of standard basis vectors is clearly orthonormal and complete in the appropriate inner product.

It is less clear why a space of functions defined on a continuous domain would be separable, and showing this is nontrivial. Let $C[a,b]$ denote the vector space of real valued continuous functions on $[a,b]$ and let $P[a,b]$ denote the subspace of polynomial functions (these are not Hilbert spaces because they are not complete). Then we have the inclusions:
$$ P[a,b] \subset C[a,b] \subset L^2[a,b],$$
as subspaces. The key is that both of these inclusions are *dense* in the $L^2$ norm; the first one is implied by the *Weierstrass Approximation Theorem*:

**Theorem 1.** Suppose $f\in C[a,b]$ and $\epsilon>0$. Then there is a polynomial $p\in P[a,b]$ such that $|f(x)-p(x)|<\epsilon$ uniformly on $[a,b]$.
Since the uniform approximation above implies closeness in $L^2$, the first inclusion follows. We will prove this theorem later in the lecture.

The second inclusion follows straightforwardly from the definition of the Lebesgue integral and measurable functions; the interested reader can consult Reed and Solomon exercise II.2.

We can now see that $L^2[a,b]$ is separable: let $S$ be an enumeration of all polynomials with rational coefficients, viewed as elements of $P[a,b]$. Then for every $\epsilon>0$, one can find $c\in C[a,b], p\in P[a,b]$, and $s\in S$ satisfying
$$ \|f-c\|<\epsilon/3\quad \|c-p\|<\epsilon/3\quad \|p-s\|<\epsilon/3,$$
all in the $L^2$ norm. Thus $\|f-s\|<\epsilon$; since $S$ is countable we are done.

To see that $L^2(\R)$ is separable, let $S_n$ be a countable dense subset of $L^2[-n,n]$ for every $n$, and set $S:=\bigcup_n S_n$. For $f\in L^2(\R)$ and $\epsilon>0$ choose $n$ large enough so that $\|f\1_{[-n,n]} - f\|<\epsilon/2$, where $\1_{[-n,n]}$ is an indicator function (such an $n$ exists because the integral is finite). Then choose $s\in S_n\subset S$ with $\|f\1_{[-n,n]}-s\|<\epsilon/2$.

Separability has an important conceptual consequence: every separable Hilbert space $H$ is *ismorphic to $\ell^2$, in the sense that the linear map
$U:H\rightarrow \ell^2$ defined by
$$U(x)=((\phi_1,y), (\phi_2,x), \ldots)$$
where $\{\phi_n\}$ is an ONB of $H$ is a Hilbert space isomorphism, i.e.,
$$( U(x),U(y))_{\ell^2} = (x,y)_H$$
for every $x,y\in H$.
Thus, every infinite dimensional countable Hilbert space is "the same" (in fact, early texts on the subject referred to it as "the" Hilbert space).
The finite dimensional ones are all isomorphic to $\mathbb{C}^n$ or $\R^n$ by elementary linear algebra.

We will henceforth assume all Hilbert spaces being discussed are separable, without mention.

## Explicit Bases

The reason we talk about "different" Hilbert spaces in practice is that the choice of basis makes a big difference with regards to understanding the structure of the space. The "generic" bases arising from separability are not very useful, and we now describe some more interesting bases.

**Legendre Polynomials.** The set of monomials $\{1,x,x^2,\ldots\}$ is complete in $L^2[-1,1]$, i.e., if $(x^n,f)=0$ for every $n$ then
$(p,f)=0$ for every $p\in P[-1,1]$. To see this, note that $(p,f)=0$ implies $\|p-f\|^2 = \|p\|^2+\|f\|^2$ for every $p$, which by the Weierstrass theorem gives $f=0$. Thus, we can
apply Gram-Schmidt to the set of monomials (with respect to the $L^2$ inner product) to obtain a set of *orthogonal polynomials* $\{L_n(x)\}_{n=1}^\infty$
which are an ONB for $L^2[-1,1]$. These particular polynomials are called Legendre polynomials (up to constant multiples; the traditional normalization does not take them to be unit vectors) and have many other nice properties which we will discuss later in the course. They also come up in physics when solving Laplace's equation in spherical coordinates.

**Fourier Series.** Another good basis for $L^2[0,2\pi]$ is given by the exponential functions $\{(2\pi)^{-1/2} e^{inx} \}_{n=-\infty}^{\infty}$; convergence of Fourier series in $L^2$ (a fact which we will not prove, though a proof is outlined in Reed and Simon II.8-9) implies that
$$\frac{1}{\sqrt{2\pi}}\sum_{n} (e^{inx},f)e^{inx} \rightarrow f$$
in $L^2$, for every $f\in L^2$.

**Hermite Functions.**
The harmonic oscillator wavefunctions, i.e., the set of solutions $\phi_n$ to
$$ (-\frac{d^2}{dx^2} + x^2)\phi_n = (2n+1)\phi_n$$
form an orthonormal basis of $L^2(\R)$. We may prove this later in the course. These functions are (up to constants) *Hermite polynomials* multiplied by a Gaussian.

The point is that a lot of good bases come from differential equations arising in physics. Later on, we will also consider questions like: how fast do the coefficients in the generalized Fourier expansions corresponding to these bases decay?

## Closed Subspaces and Projections

The subspaces $P[a,b], C[a,b]$ in the previous sections are vector spaces, but they are not Hilbert spaces because they are not closed under taking limits; they are examples of *dense subspaces*.

**Definition.** A *closed subspace* $M$ of a Hilbert space $H$ is a subspace which is closed under taking limits with respect to $\|\cdot\|$.
The *orthogonal complement* of a subspace is defined as:
$$ M^\perp := \{ x\in H: (x,y)=0\quad\forall y\in M\}.$$
It is easy to check that the orthogonal complement of a subspace (closed or not) is closed.

Closed subspaces are Hilbert spaces in their own right, and satisfy the following important theorem.
**Theorem 2.** If $M$ is a closed subspace of $H$, then every vector $x\in H$ can be *uniquely* decomposed as
$$ x = y + z$$
where $y\in M$ and $z\in M^\perp$.
*Proof.* Since $M$ is a separable Hilbert space, it has an ONB $\{\phi_n\}$. Let
$$ y = \sum_{n} (\phi_n,x)\phi_n,$$
noting that the sum converges in $H$ by Bessel's inequality, and $y\in M$ since $M$ is complete. We now have
\begin{align*}
(x-y,y) &= (x,y)-\|y\|^2 \\
&= (x,\sum_n (\phi_n,x)\phi_n)-\|y\|^2\\
&= \sum_n (\phi_n,x)(x,\phi_n)-\|y\|^2\quad\textrm{by continuity of $(\cdot,\cdot)$}\\
&= \sum_n |(\phi_n,x)|^2-\|y\|^2\\
&=0,
\end{align*}
by Parseval. Thus $x-y\in M^\perp$. To see uniqueness, observe that $x=y'+z'$ for any $y',z'\in M,M^\perp$ implies that $y$ and $y'$ have the same Fourier coefficients with respect to $\phi_n$, which means they are equal. $\square$

It can be easily shown using calculus that $y$ is the point in $M$ minimizing $\|x-y\|$. One pleasing application of this is that Hilbert space theory gives an algorithm for finding the best low degree polynomial approximation to a given $L^2$ function in the $L^2$ norm: simply compute the first few Legendre coefficients by integration, and output the corresponding sum of Legendre polynomials.

**Remark** A closed subspace $M\subset B$ of a Banach space is *complemented* if there is another subspace $C\subset B$ such that every $x\in B$ can be uniquely decomposed as $y+z$ with $y\in M, z\in C$; this is denoted $B=M\oplus C$. We have just shown that every closed subspace of a Hilbert space is complented. IT can be shown that Hilbert space is the *only* Banach space with this property.

## Proof of Weierstrass Theorem

See http://people.math.sc.edu/schep/weierstrass.pdf

## Dual Space

A *linear functional* on $H$ is a linear map $\ell:H\rightarrow \mathbb{C}$. The *norm* of a linear functional is
$$ \|\ell\| = \sup_{x\in H\setminus \{0\}} \|\ell(x)|/\|x\|,$$
and it is called bounded if it has finite norm.
**Proposition.** A linear function is continuous (with respect to convergence in $H$) if and only if it is bounded. (proof on the homework)

Note that whenever we mention linear functions that are continuous, we must be mindful of which notion of convergence they are continuous with respect to; we will consider various other notions later in the course. Though we will not explicitly mention topological spaces, they are essentially an abstract way of specifying a notion of convergence.

The set of bounded linear functions on $H$ forms a vector space with respect to pointwise addition and multiplication, caled the dual space $H^*$. In fact, there is an isometry (norm-preserving bijection) between $H$ and $H^*$.

**Theorem.**(Riesz-Fisher) For every $\ell\in H^*$ there is a unique $y\in H$ such that $\ell(x)=(y,x)$ for all $x\in H$. This map is an *isometry*, i.e., a bijection satisfying $\|\ell\|_{H^*}=\|y\|_H$.

The proof is given in Reed and Simon Theorem II.4. In particular, this implies that $H^*$ is also a Hilbert space.