Linear Analysis¶

$\ell_2^n \coloneqq (\mathbb{R}^n, \lVert \cdot \rVert_2)$ (or $(\mathbb{C}^n, \lVert \cdot \rVert_2)$), where $$\lVert x \rVert_2 \coloneqq \left(\sum_{i=1}^n |x_i|^2 \right)^{1/2}$$ for $x = (x_1, \dots, x_n) \in \mathbb{R}^n$ (the $\ell_2$-norm or Euclidean norm).

Check the three properties: (i), (ii) are easy; (iii) follows from Cauchy–Schwarz.

$\ell_1^n \coloneqq (\mathbb{R}^n, \lVert \cdot \rVert_1)$, where $\lVert x \rVert_1 \coloneqq \sum_{i=1}^n |x_i|$ (the $\ell_1$-norm).

(i), (ii) easy; (iii): $|x_i + y_i| \leq |x_i| + |y_i|$, sum over $i$.

$\ell_{\infty}^n \coloneqq (\mathbb{R}^n, \lVert \cdot \rVert_{\infty})$, where $\lVert x \rVert_{\infty} \coloneqq \max_{1 \leq i \leq n} |x_i|$ (the $\ell_{\infty}$-norm or sup-norm).

(i), (ii), (iii) easy.

$\ell_p^n \coloneqq (\mathbb{R}^n, \lVert \cdot \rVert_p)$, where $1 \leq p < \infty$, $\lVert x \rVert_p \coloneqq \left(\sum_{i=1}^n |x_i|^p\right)^{1/p}$ (the $\ell_p$-norm).

(i), (ii) easy, (iii) not obvious (done later; see ?).

Let $S$ denote the set of all scalar sequences. This is a vector space in the coordinate-wise operations: $(x_n) + (y_n) = (x_n + y_n), \lambda \cdot (x_n) = (\lambda x_n)$.

$\ell_1 \coloneqq \Big\{(x_n) \in S \;\Big|\; \sum_{n=1}^{\infty} |x_n| < \infty\Big\},\; \lVert (x_n) \rVert_1 \coloneqq \sum_{n=1}^\infty |x_n|$ (the $\ell_1$-norm).

(i), (ii) easy. (iii): given $x = (x_n), y = (y_n) \in \ell_1$,

• $|x_n + y_n| \leq |x_n| + |y_n| \;\forall\; n \in \mathbb{N}$.
• Sum over $n \in \mathbb{N}$ to get that $x + y \in \mathbb{\ell_1}$ and $\lVert x + y \rVert_1 \leq \lVert x \rVert_1 + \lVert y \rVert_1$.

So $\ell_1$ is a subspace of $S$ and $\lVert \cdot \rVert_1$ is a norm on $\ell_1$.

$\ell_2 \coloneqq \Big\{(x_n) \in S \;\Big|\; \sum_{n=1}^{\infty} |x_n|^2 < \infty\Big\},\; \lVert (x_n) \rVert_2 \coloneqq \left(\sum_{n=1}^\infty |x_n|^2 \right)^{1/2}$ (the $\ell_2$-norm).

(i), (ii) easy. (iii): given $x = (x_n), y = (y_n) \in \ell_2$,

• $\left(\sum_{k=1}^n |x_k+y_k|^2\right)^{1/2} \leq \left(\sum_{k=1}^n |x_k|^2\right)^{1/2} + \left(\sum_{k=1}^n |y_k|^2\right)^{1/2}$ by triangle inequality in $\ell_2^n$.
• Let $n \to \infty$: get that $x + y \in \mathbb{\ell_2}$ and $\lVert x + y \rVert_2 \leq \lVert x \rVert_2 + \lVert y \rVert_2$.

So $\ell_2$ is a subspace of $S$ and $\lVert \cdot \rVert_2$ is a norm on $\ell_2$.

More generally, for $1 \leq p < \infty$, $\ell_p \coloneqq \Big\{(x_n) \in S \mid \sum_{n=1}^\infty |x_n|^p < \infty \Big\}$ is a subspace of $S$ and $\lVert (x_n) \rVert_p \coloneqq \left(\sum_{n=1}^\infty |x_n|^p \right)^{1/p}$ is a norm (the $\ell_p$-norm) on $\ell_p$ (once we have the triangle inequality in $\ell_p^n$, which we will do soon; see ?).

$\ell_\infty \coloneqq \Big\{(x_n) \in S \;\Big|\; \exists M \geq 0 \quad \forall n \quad |x_n| \leq M \Big\},\; \lVert (x_n) \rVert_\infty \coloneqq \sup_{n \in \mathbb{N}} |x_n|$ (the $\ell_\infty$-norm or sup-norm).

(i), (ii) easy. (iii): given $x = (x_n), y = (y_n) \in \ell_\infty$,

• $|x_n + y_n| \leq |x_n| + |y_n| \leq \lVert x \rVert_\infty + \lVert y \rVert_\infty \;\forall\; n \in \mathbb{N}$.
• So $x+y \in \ell_\infty$ and $\lVert x+y \rVert_\infty \leq \lVert x \rVert_\infty + \lVert y \rVert_\infty$.

$c_0 \coloneqq \Big\{(x_n) \in S \;\Big|\; x_n \to 0 \text{ as } n \to \infty \Big\}$

$c \coloneqq \Big\{(x_n) \in S \;\Big|\; \lim_{n \to \infty} x_n \text{ exists} \Big\}$

These are subspaces of $\ell_\infty$ and hence normed spaces in $\lVert \cdot \rVert_\infty$.

$C[0,1] \coloneqq \big\{f : [0,1] \to \mathbb{R} \mid f \text{ is continuous}\big\}$, $\lVert f \rVert_\infty \coloneqq \sup_{x \in [0,1]} |f(x)|$ (sup-norm or uniform norm)

This is a Banach space (recall uniform limit of cts fns is cts).

More generally, given a compact Hausdorff space $K$, $$C(K) \coloneqq \big\{f:K\to\mathbb{R} \mid f \text{ cts}\big\}$$ is a Banach space in the sup-norm $\lVert f \rVert_\infty \coloneqq \sup_{x \in K} |f(x)|$. (We will look at compact Hausdorff spaces in depth later.)

$C[0,1]$ with the $L^1$-norm $\lVert f \rVert_1 \coloneqq \int_0^1 |f(t)| dt, f \in C[0,1]$

This is a normed space, but is incomplete:

More generally, $C[0,1]$ is incomplete in the $L^p$-norm, $1 \leq p < \infty$, $\lVert f \rVert_p \coloneqq \left(\int_0^1 |f(t)|^p dt\right)^{1/p}$ (same counterexample).

$\big[$ In II Prob & Measure, the completion of $\left(C[0,1], \lVert \cdot \rVert_p\right)$ is $L^p[0,1]$ $\big]$

$C^1[0,1] \coloneqq \big\{ f \in C[0,1] \mid f \text{ ctsly diffble} \big\} \subseteq C[0,1]$ is a subspace. So it's a normed space in $\lVert \cdot \rVert_\infty$, but is incomplete, i.e. not closed in $C[0,1]$.

It is complete in the norm $\lVert f \rVert = \lVert f \rVert_\infty + \lVert f' \rVert_\infty$ (see example sheet 1).

$\Delta \coloneqq \big\{ z \in \mathbb{C} \mid |z| \leq 1 \big\}$

$A(\Delta) \coloneqq \big\{ f \in C(\Delta) \mid f \text{ analytic on int}(\Delta) \big\}$ is a subspace of $C(\Delta)$, and a closed space [uniform limit of holomorphic fns is holomorphic], hence a Banach space in $\lVert \cdot \rVert_\infty$.

$T: \ell_2^n \to \ell_2^n,\; T(x_1, \dots, x_n) = (x_1, \dots, x_r, 0, \dots, 0), 1 \leq r \leq n$.

$\lVert Tx \rVert_2 = \left(\sum_{i=1}^r |x_i|^2\right)^{1/2} \leq \lVert x \rVert_2$. So $\lVert T \rVert \leq 1$.

But $Te_1 = e_1$, so $\lVert T \rVert = 1$.

More generally, if $T$ has matrix $A = (a_{ij})$ w.r.t. standard basis, then $\lVert T \rVert \leq \left(\sum_{i,j=1}^n |a_{ij}|^2\right)^{1/2}$ by Cauchy-Schwarz.

Let $1 \leq p \leq \infty,\; R: \ell_p \to \ell_p, R(x_1, x_2, \dots) = (0, x_1, x_2, \dots)$ (the right-shift).

$\forall x \in \ell_p, \lVert Rx \rVert_p = \lVert x \rVert_p$. So $R$ is isometric and $\lVert R \rVert = 1$.

$R$ is injective but not surjective.

Let $1 \leq p \leq \infty,\; L: \ell_p \to \ell_p, L(x_1, x_2, \dots) = ((x_2, x_3, \dots)$ (the left-shift).

$\forall x \in \ell_p, \lVert Rx \rVert_p \leq \lVert x \rVert_p$. So $L \in \mathcal{B}(\ell_p), \lVert L \rVert \leq 1$. As $Le_2 = e_1, \lVert L \rVert = 1$.

$L$ is surjective but not injective; $LR = Id \neq RL$.

$T : \ell_1 \to \ell_2, Tx = x$.

Claim: $\ell_1 \subseteq \ell_2$ and for $x \in \ell_1, \lVert x \rVert_2 \leq \lVert x \rVert_1$. Indeed, WLOG $\lVert x \rVert_1 = 1$ by homogeneity, so $\sum |x_i| = 1$ and $|x_i| \leq 1 \;\forall\; i$; hence $|x_i|^2 \leq |x_i| \forall i$, so $\lVert x \rVert_2^2 \leq \lVert x_1 \rVert = 1$.

Thus $T \in \mathcal{B}(\ell_1, \ell_2)$ and $\lVert T \rVert = 1$ (e.g. $Te_1 = e_1, \lVert e_1 \rVert_2 = \lVert e_1 \rVert_1 = 1).$

$T: \ell_2 \to \ell_1, T((x_n)) = \left(\frac{x_n}{n}\right).$ By Cauchy-Schwarz, $\sum \frac{|x_n|}{n} \leq \left(\sum |x_n|^2\right)^{\frac{1}{2}} \left(\sum \frac{1}{n^2}\right)^{\frac{1}{2}}$

So $T \in \mathcal{B}(\ell_2, \ell_1)$ and $\lVert T \rVert \leq \left( \sum \frac{1}{n^2} \right)^{\frac{1}{2}}$; in fact we have equality by equality in C-S (set $x_n = \frac{1}{n}$)

$D : \left(C^1[0,1], \lVert \cdot \rVert\right) \to \left(C[0,1], \lVert \cdot \rVert_\infty \right), Df = f'$.

$D$ is bounded: $\lVert Df \rVert_\infty = \lVert f' \rVert_\infty \leq \lVert f' \rVert_\infty + \lVert f \rVert_\infty = \lVert f \rVert$. So $\lVert D \rVert \leq 1$.

To show equality, take $f(x) = \sin(n \pi x)$, then $\lVert Df \rVert_\infty = \pi n, \lVert f \rVert = \pi n + 1$. Thus $\lVert D \rVert = 1$ (take $n$ arb. large).

For $f \neq 0, \lVert Df \rVert < \lVert f \rVert$. But still, $\lVert D \rVert = 1$.

On a normed space $X$, the identity $x \mapsto x$ is denoted by $Id, I, Id_X$ or $I_X$. This is isometric i.e. $\lVert Id(x) \rVert = \lVert x \rVert \;\forall x.$

For normed spaces $X,Y$, we let $$X \oplus Y \coloneqq \big\{ (x,y) \mid x \in X, y \in Y\big\}$$ with norm $\lVert (x,y) \rVert_1 \coloneqq \lVert x \rVert + \lVert y \rVert$. The corresponding norm topology is the product topology.

If $X \sim Y$, then $X$ complete $\iff Y$ complete.

If $\lVert \cdot \rVert, \lVert \cdot \rVert'$ are equivalent norms on a vector space $X$, then $(X,\lVert \cdot \rVert)$ is complete iff $(X, \lVert \cdot \rVert')$ is complete.

(Isomorphisms preserve completeness, since bilipschitz bijection)

Given normed spaces $X,Y$, on $X \oplus Y$ the norm $\lVert (x,y) \rVert_1 = \lVert x \rVert + \lVert y \rVert$ is equivalent to $$\lVert (x,y) \rVert_p \coloneqq \left(\lVert x \rVert_p + \lVert y \rVert_p \right)^{\frac{1}{p}} \; (1 \leq p < \infty)$$ $$\text{\& to } \lVert (x,y) \rVert_\infty \coloneqq \max\{\lVert x \rVert, \lVert y \rVert\}$$

On $C[0,1]$, $\lVert \cdot \rVert_\infty$ is complete and $\lVert \cdot \rVert_1$ is incomplete. So $\lVert \cdot \rVert_\infty \nsim \lVert \cdot \rVert_1$. (There does exist an easy direct proof)

However $\lVert f \rVert_1 = \int_0^1 |f(t) dt \leq \lVert f \rVert_\infty$.

So $Id: (C[0,1], \lVert \cdot \rVert_\infty \to (C[0,1], \lVert \cdot \rVert_1)$ is a cts linear bijection, but its inverse is not cts.

On $c_{00} = \text{span}\{e_n \mid n \in \mathbb{N}\}, \lVert \cdot \rVert_1 \nsim \lVert \cdot \rVert_2$:

Take $x = (1, 1, \dots, 1, 0, \dots)$, then $\lVert x \rVert_1 = n, \lVert x \rVert_2 = \sqrt n$.

$(Id_X)^* = Id_{X^*}$

$(\lambda S + \mu T)^* = \lambda S^* + \mu T^* \;(S,T \in \mathcal{B}(X,Y), \; \lambda,\mu \text{ scalars})$

Indeed, for $g \in Y^*, x \in X,$

$$\begin{aligned} \langle x,(\lambda S + \mu T)^* g\rangle = \langle (\lambda S + \mu T)x,g\rangle &= \langle\lambda Sx + \mu Tx,g\rangle\\ &= \lambda \langle Sx,g\rangle + \mu\langle Tx,g\rangle\\ &= \lambda \langle x,S^*g\rangle + \mu\langle x,T^*g\rangle\\ &= \langle x,(\lambda S^* + \mu T^*)g\rangle \end{aligned}$$

Since $x$ arbitrary, have $(\lambda S + \mu T)^* g = (\lambda S^* + \mu T^*)g$

Since $g$ arbitrary, have $(\lambda S + \mu T)^* = \lambda S^* + \mu T^*$, as required.

$(ST)^* = T^* S^* \;(T \in \mathcal{B}(X,Y), S \in \mathcal{B}(Y,Z))$

Indeed, $\langle x,(ST)^* g\rangle = \langle (ST)x,g\rangle = \langle S(Tx),g\rangle = \langle Tx,S^* g\rangle = \langle x,T^*(S^* g)\rangle = \langle x,(T^*S^*) g\rangle \;\checkmark$

Let $T \in \mathcal{B}(X,Y)$. We have $T^* \in \mathcal{B}(Y^*,X^*)$ and $T^{**} = (T^*)^* \in \mathcal{B}(X^{**},Y^{**})$.

The diagram commutes, i.e. $\widehat{Tx} = T^{**}\hat{x} \;\forall x \in X$.

Indeed, for $x \in X$ and $g \in Y^*,\quad\langle g,T^{**}\hat{x}\rangle = \langle T^* g,\hat{x}\rangle = \langle x,T^*g\rangle = \langle Tx,g\rangle = \langle g,\widehat{Tx}\rangle\;\checkmark$