$\newcommand{\ds}{\displaystyle}$ $\newcommand{\Rr}{\mathbb{R}}$ $\newcommand{\Qq}{\mathbb{Q}}$ $\newcommand{\Nn}{\mathbb{N}}$ $\newcommand{\lint}{\underline{\int}}$ $\newcommand{\uint}{\overline{\int}}$ $\newcommand{\sgn}{\mathrm{sgn}}$ $\newcommand{\ve}{\varepsilon}$ $\newcommand{\norm}[1]{\left\| #1 \right\|}$

Sequences of Functions

Pointwise and Uniform convergence

A sequence $(f_n)$ of functions converges pointwise on $S$ to a function $f$ if $f_n(x) \to f(x)$ (as $n \to \infty$) for every $x \in S$.

We called $f$ a limit function of the sequence $(f_n)$. By uniqueness of limits for sequences, if $(f_n)$ has a limit function on $S$, then it has a unique limit function on $S$.

Exercise. Write down an $\varepsilon-N$ definition of pointwise convergence (see. Proposition 6.1.5 in the text for an answer).

Example. Let $f_n(x) = x^{2n}$. The sequence of functions $(f_n(x)) = x^2,x^4,x^6, \cdots$ converges pointwise on $[-1,1]$ to the function $$ f(x) = \begin{cases} 1 & x = \pm 1\\ 0 & x \in (-1,1) \end{cases} $$

Note that even each $f_n(x)$ is a "nice" (polynomial) function of $x$ on the real line, the sequence only converges on $[-1,1]$ and that the limit function $f(x)$ is not continuous on $[-1,1]$.

This example shows that many properties of functions are not preserved by taking pointwise limit.

Example. A convergent power series $\sum a_n(x-x_0)^n$ is the limit function of $(s_k(x))$ where $s_k(x) = \sum_{n=0}^k a_n(x-x_0)^n$ on some neighborhood of $x_0$.

For example, the sequence of polynomials $$ 1, 1+x, 1+x+x^2, \cdots, \sum_{n=0}^k x^n, \cdots $$ converges to the function $\dfrac{1}{1-x}$ on $(-1,1)$.

Example. Discuss the convergence of the sequence $(\sin(nx))$ read Example 6.1.4 in the textbook.

Uniform convergence

Let $f,f_n$ $(n \in \Nn)$ be functions defined on $S$.

We say that $(f_n)$ converges to $f$ uniformly on $S$ if

for any $\varepsilon >0$, there exists $N=N(\varepsilon) \in \mathbb{N}$ such that for all $n \ge N$ and $x \in S$, $$ |f_n(x)-f(x)| \lt \varepsilon. $$

A sequence of functions on $S$ is uniformly convergent if it uniformly converges to some function on $S$.

N.B. The $N$ appears in the definition above depends only on $\varepsilon$ and dose not depend on the point $x \in S$.

It is clear from the definition that

Proposition 1. Uniform convergence implies pointwise convergence.

More precisely, if $(f_n)$ converges to $f$ on $S$ uniformly then it pointwise converges to $f$.

The converse of this proposition is not true.

Example. The sequence of functions $f_n(x) = x^{2n}$ converges pointwise (as we have seen) but not uniformly on $[-1,1]$.

To see this, suppose on the contrary that the convergence is uniformly. Then for $\ve = 1/2$, there exists some $N \in \Nn$, such that for all $x \in [-1,1]$, $|x^{2N} - f(x)| \lt 1/2$. Since $f(x) =0$ for $x\neq \pm 1$. So, for any $-1 \lt x \lt 1$,

$$ |x^{2N} - f(x)| = |x^{2N}-0| = x^{2N} \lt \frac{1}{2}. \tag{*}$$

But since $1-1/k \in (-1,1)$ for any $k \in \Nn$, so by ($*$), $(1-1/k)^{2N} < 1/2$. Letting $k \to \infty$, it follows from the continuity of $f_N(x) = x^{2N}$ that

$$1 = f_N(1) = \lim_k f_N(1-1/k) = \lim_k (1-1/k)^{2N} \le \lim_k 1/2 = 1/2.$$

That is cleary absurd.

Sup-norm (aka uniform norm)

Let $V$ be a $\Rr$-vector space. A norm on $V$ is a function $\norm{\cdot} \colon V \to \Rr$ such that

1. $\norm{x+y} \le \norm{x} + \norm{y}$ (Subadditive)
2. $\norm{cx} = |c|\norm{x}$ (absolute homogeneity)
3. $\norm{x} = 0$ only if $x$ is the zero vector. (positive definite)

It follows that

$$ \norm{\mathbf{0}} = \norm{x +(-x)} \le \norm{x} + \norm{-x} = \norm{x} + |-1|\norm{x} = 2\norm{x} $$

In particular, when $x=\mathbf{0}$, we see that $0 \le \norm{\mathbf{0}}$.

Therefore, $0 = \norm{\mathbf{0}} \le 2\norm{x}$ and hence $\norm{x} \ge 0$ for every $x \in V$. Together with (3), we conclude that $\norm{x} > 0$ for all $x \neq \mathbf{0}$.

The relevent norm for studying uniform convergence is the sup-norm.

Let $f \colon S \to \mathbb{R}$ be bounded. The sup-norm of $f$ on $S$ is defined to be

$$\|f\|_S = \sup\{|f(x)| \colon x \in S\}.$$

We often write $\| \cdot \|$ instead of $\| \cdot \|_S$ if $S$ is understood.

Intuitively, $\|f\|$ should be regarded as the 'size' of the function $f$, e.g. the length of a vector.

Exercise. Show that if $(f_n)$ is a sequence of bounded functions on $S$ and $(f_n)$ converges to $f$ uniformly on $S$ that $f$ is also bounded on $S$.

Proposition 2. Suppose $(f_n)$ is a sequence of functions on $S$ and $f$ is a function on $S$. Then

$f_n \to f$ uniformly on $S$ if and only if $\|f_n-f\|_S \to 0$.

In this case, we also say that the sequence of functions $(f_n)$ converges in sup-norm to $f$.

Proof. Suppose $f_n \to f$ uniformly on $S$, then for any $\ve >0$, there exists $N \in \mathbb{N}$,

such that for any $x \in S$ and any $n \ge N$, $|f_n(x)-f(x)| < \ve$.

Thus, for any $n \ge N$, $$ \norm{f_n-f} = \sup\{|f_n(x)-f(x)| \colon x \in S\} \le \ve. $$

Since $\ve > 0$ is arbitrary, this shows that $\norm{f_n-f} \to 0$ as $n \to \infty$.

Conversely, suppose $\norm{f_n-f} \to 0$. So, for any $\ve > 0$, there exists $N \in \Nn$ such that $\norm{f_n-f} < \ve$ for all $n \ge N$. Therefore,

$$ |f_n(x)-f(x)| \le \sup\{|f_n(x)-f(x)| \colon x \in S\} = \norm{f_n-f} < \ve.$$

That means $f_n \to f$ uniformly on $S$.

Remark Note that we do not need to assume the functions $f_n$'s or the function $f$ are bounded on $S$ in this proposition.

Read Example 6.1.11 from the textbook.

The sup-norm can be regarded as a generalization of the absolute value and one can talk about Cauchy sequence in sup-norm.

A sequence of function $(f_n)$ is uniformly Cauchy (or Cauchy with respect to the sup-norm)

if for every $\varepsilon \gt 0$, there exists $N$, such that for all $m, k \ge N$, $$ \| f_m - f_k\| \lt \varepsilon. $$

As one can expect from the completeness of the real numbers that

Proposition (6.1.13). Let $(f_n)$ be a sequence of functions on $S$. Then

$(f_n)$ is uniformly Cauchy on $S$ if and only if $f_n \to f$ uniformly on $S$ for some function $f$ on $S$.

Proof. Suppose $(f_n)$ converges uniformly on $S$ to some $f$. Then for any $\ve > 0$, $\norm{f_n-f} < \ve/2$ for all sufficiently large $n$. In particular,

$$ \norm{f_m-f_k} \le \norm{f_m-f} + \norm{f_k -f} < \ve$$

for all $m,k$ sufficiently large. Thus, $(f_n)$ is uniformly Cauchy.

Conversely, suppose $(f_n)$ is uniformly Cauchy. Then for any $x \in S$ $$ |f_m(x)-f_k(x)| \le \norm{f_m-f_k}, $$ so $(f_n(x))$ is a Cauchy sequence of real number for any $x \in S$. By completeness, $(f_n(x))$ converges for every $x \in S$.

So the function $f(x) = \lim_n f_n(x)$ ($x \in S$) exists and $f_n \to f$ pointwise.

To see that the convergence is uniform on $S$, note that for any $\ve >0$, since $(f_n)$ is uniformly Cauchy there exists $N \in \mathbb{N}$ such that for any $\norm{f_m-f_k} \le \ve $ whenever $m,k \ge N$. Thus, for all $x \in S$,

$$ |f_m(x) - f_k(x)| \le \norm{f_m - f_k} < \ve.$$

Fixing $m \ge N$ and letting $k \to \infty$, we conclude that for any $x \in S$,

$$ |f_m(x) - f(x)| \le \ve$$

Therefore, $f_n \to f$ uniformly on $S$.

The next result, in many cases, is an easy to use test for uniform convergence.

Theorem (Weierstrass M-test)

Suppose $(f_k)$ is a sequence of functions on $S$ and $(M_k)$ is a sequence of non-negative numbers such that

1. $\norm{f_k} \le M_k$ for each $k$ and
2. $\sum M_k$ converges

then $s_n(x) = \sum_k^n f_k(x)$ converges absolutely and uniformly on $S$.

Proof. Since $\norm{f_k} = \norm{|f_k|}$ condition 1. is unchanged if $(f_k)$ is replaced by $(|f_k|)$, so absolute convergence follows immediately from pointwise convergence. And since pointwise convergence follows from uniform convergence, all we need to show is that the series $\sum_k f_k(x)$, i.e. the sequence of partial sums $(s_n(x))$, converges uniformly on $S$.

The series of numbers $\sum M_k$ converges (assumption (2)). So its sequence of partial sums is Cauchy.

Thus, for any $\ve > 0$, there exists $N \in \Nn$ such that $\sum_{k=n}^m M_k < \ve$ whenever $m > n \ge N$.

Let $s_n(x) = \sum_{k=1}^n f_k(x)$ be the $n$-th partial sum of the series $\sum f_k(x)$. Then for any $m > n \ge N$, it follows from the subadditivity of norm and assumption (1), that

$$ \norm{s_n(x)-s_m(x)} = \norm{\sum_{k=n+1}^m f_k(x)} \le \sum_{k=n+1}^m \norm{f_k(x)} \le \sum_{k=n+1}^m M_k < \ve$$

But that means the sequence of functions $(s_n(x))$ is uniformly Cauchy on $S$ and so uniformly converges by Proposition 6.1.13 (above).