$\newcommand{\ds}{\displaystyle}$ $\newcommand{\Rr}{\mathbb{R}}$ $\newcommand{\Qq}{\mathbb{Q}}$ $\newcommand{\Nn}{\mathbb{N}}$ $\newcommand{\lint}{\underline{\int}}$ $\newcommand{\uint}{\overline{\int}}$ $\newcommand{\sgn}{\mathrm{sgn}}$ $\newcommand{\ve}{\varepsilon}$ $\newcommand{\norm}[1]{\left\| #1 \right\|}$

Interchange of limits

In general, the order of taking limits cannot be interchanged as the following example shows

$$ 0 = \lim_{n}\left(\lim_{k} \frac{n/k}{n/k+1}\right) \neq \lim_{k}\left(\lim_{n} \frac{n/k}{n/k+1}\right) = 1. $$

Exercise. Let $f(x,y) = \dfrac{x}{x+y}$.

Find $$ \lim_{x \to 0}\lim_{y \to 0} f(x,y) \quad \text{and} \quad \lim_{y \to 0}\lim_{x \to 0} f(x,y). $$

Continuity of the limit

The pointwise limit the sequence of continuous functions $(x^n)$ on $[0,1]$) is the function $$ f(x) = \begin{cases} 0 & 0 \le x < 1 \\ 1 & x =1, \end{cases} $$

and it is not continuous. This provides another example that the order of taking limits cannot be switched.

$$ \lim_{x \to 1} \lim_n x^n = \lim_{x \to 1} f(x) = 0.$$

$$ \lim_n \lim _{x \to 1} x^n = \lim_n 1^n = 1.$$

Read Example 6.2.1 in the textbook for another example.

However, if the convergence is uniform, the limit processes above can be interchanged. More precisely, we have

Theorem 6.2.2 Suppose $f_n(x)$ is continuous at $x_0 \in S$ for each $n$ and $f_n \to f$ uniformly on $S$ then $f$ is continuous at $x_0$.

Proof. For any $\ve >0$, there exists $N=N(\ve) \in \Nn$ such that $$\norm{f_N-f} < \ve/3 \tag{1}$$

By assumption, $f_N$ is continuous at $x_0$ so there exists $\delta >0$ (depending on $\ve$ and $x_0$) such that whenever $x \in S$ and $|x-x_0| < \delta$,

$$ |f_N(x)-f_N(x_0)| < \ve /3. \tag{2}$$

It follows from (1) and (2) that $$ \begin{align*} |f(x)-f(x_0)| &\le |f(x)-f_N(x)| + |f_N(x)-f_N(x_0)| + |f_N(x_0) -f(x_0)| \\ &\le \norm{f-f_N} + |f_N(x)-f(x_0)| + \norm{f-f_N} < \ve/3 + \ve/3 + \ve/3 = \ve \end{align*} $$ whenever $x \in S$ and $|x-x_0| < \delta$. This shows that $f$ is continuous at $x_0$.

Corollary The uniform limit of a sequence $(f_n)$ of continuous functions on $S$ is continuous on $S$.

Exercise. Show that uniform continuity is preserved by uniform convergence.

That is shows that the uniform limit of a sequence of uniformly continuous functions on a set $S$ is uniformly continuous.

Example. Consider the sequence of functions $(f_n)$ on $[0,1]$ where $f_n = \sqrt{x+1/n}$. For each $n$, and for all $s,t \in [0,1]$,

$$|f_n(s)-f_n(t)| = \frac{|(s+1/n) - (t+1/n)|}{\sqrt{s+1/n} + \sqrt{t+1/n}} \le \frac{|s-t|}{2/\sqrt{n}} = \frac{\sqrt{n}}{2}|s-t|.$$

This shows that $f_n$ is Lipschitz continuous on $[0,1]$. The limit function of this sequence is $f(x)=\sqrt{x}$.

For any $x \in [0,1]$,

$$ |f_n(x) - f(x)| \le \frac{x+(1/n)-x}{\sqrt{x+1/n} +\sqrt{x}} \le \frac{1}{\sqrt{n}}$$

Therefore, $\norm{f_n-f} \le 1/\sqrt{n} \to 0$. This shows that $f_n \to f$ uniformly on $[0,1]$.

However, $f(x) = \sqrt{x}$ is not Lipschitz continuous on $[0,1]$ because

$$ \frac{|f(x)-f(0)|}{|x-0|} = \frac{1}{\sqrt{x}}\to +\infty, \text{as } x \to 0.$$

This examples shows that Lipschitz continuity is not preserved under uniform limit.

Integral of the limit

Pointwise limit also does not preserve integrals.

Example. Consider the sequence of functions $(f_n)$ on $[0,1]$ defined by

$$ \begin{align*} f_n(x) = \begin{cases} 4n^2x & 0 \le x \le 1/(2n) \\ 4n-4n^2x & 1/(2n) < x \le 1/n \\ 0 & 1/n < x \le 1. \end{cases} \end{align*} $$

Each $f_n(x)$ is continuous (hence integrable) and $\int_{0}^1 f_n=1$. Click here to see how the graphs of $f_n$ look like.

Since each positive $x$ is larger than some $1/n$, so $f_n(x) \to 0$ for each $x \in (0,1]$. Moreover, $f_n(0) = 0$ for each $n$, thus $(f_n)$ converges pointwise to the zero function $f$ on $[0,1]$. But that means

$$ 0 = \int_0^1 f = \int_0^1 \lim_n f_n \neq \lim_n \int_0^1 f_n = \lim_n 1 = 1. $$

Again if the convergence is uniform, the limit function of a sequence of integrable functions will be integrable and its integral is the limit of the integrals of functions in the sequence.

Theorem 6.2.4 Let $(f_n)$ be a sequence of integrable functions on $[a,b]$ converging uniformly to $f$. Then $f$ is integrable on $[a,b]$, and

$$\int_a^b f = \lim_n \int_a^b f_n.$$

Proof. As integrable functions are bounded, it follows that $f$ is bounded as well (as exercise in Notes 08).

By uniform convergence, for any $\ve > 0$, there exists $N$ such that $$ f_n(x) - \frac{\ve}{(b-a)} < f(x) < f_n(x) + \frac{\ve}{(b-a)}\quad \forall n \ge N. \tag{*}$$

Taking the upper integrals of the functions in (*), and using the fact the each $f_k$ is integrable, we get

$$ -\ve + \int_a^b f_n = -\ve + \overline{\int_a^b} f_n < \overline{\int_a^b} f < \ve + \overline{\int_a^b} f_n = \ve + \int_a^b f_n. \tag{1}$$

Thus, $\ds \left|\overline{\int_a^b} f - \int_a^b f_n\right| < \ve$ for all $n \ge N$. And that means $\ds \int_a^b f_n \to \overline{\int_a^b} f$ as $n \to \infty$.

Likewise, by taking the lower integrals of the functons in (*), we get

$$ -\ve + \int_a^b f_n = -\ve + \underline{\int_a^b} f_n < \underline{\int_a^b} f < \ve + \underline{\int_a^b} f_n = \ve + \int_a^b f_n. \tag{2}$$

And so, $\ds \int_a^b f_n \to \underline{\int_a^b} f$. By uniqueness of limit, the upper and the lower integral of $f$ agree and hence $f$ is integrable. Moreover their common value, i.e. the integral of $f$ over $[a,b]$ is the limit of the integrals $\ds \int_a^b f_n$.

Example 6.2.6 textbook For each $n$, the $f_n$ be the function on $[0,1]$ defined by

$$ f_n(x) = \begin{cases} 1 & x=\dfrac{p}{q}, \gcd(p,q)=1\ \text{and}\ q \le n \\ 0 & \text{otherwise}\end{cases} $$

Then each $f_n$ differ from the zero function at only finitely many places hence integrable. Moreover $\int_0^1 f = \int_0^1 0 = 0$. However, the pointwise limit of $(f_n)$ is the Dirichlet function which is not even integrable.

Example (c.f. Example 6.2.7 in textbook) For each $n$, the function $f_n$ on $[0,1]$ defined by

$$f_n(x) = \begin{cases} n^2x & 0 \le x < 1/n \\ 1/x & 1/n \le x \le 1 \end{cases}$$

is continuous (and hence bounded) on $[0,1]$. And yet, its pointwise limit is the function

$$ f(x) = \begin{cases} 0 & 0 x=0 \\ 1/x & 0< x < 1 \end{cases}$$

which is not even bounded on $[0,1]$.

However, uniform convergence is not strong enough to guarantee the sequence of derivatives converges pointwise to the derivative of the limit.

Example 6.2.8. Let $f_n(x) = \sin(nx)/n$. Then $\|f_n\| = 1/n \to 0$ and so $(f_n)$ converges uniformly to the zero function on $\mathbb{R}$. However, the derivatives $f_n'(x) = \cos(nx)$ certainly not even pointwise converges, e.g. at $x = \pi$, $\cos(n\pi) = (-1)^n$ does not converge as $n \to \infty$.

Furthermore, even if $f_n'(x)$ does converge at a point the limit may not be the derivative of the limit function at that point, e.g. $f_n'(0) = \cos(n\cdot 0) =1 \not\to 0$,

So when will the process of taking derivative and the process of taking limit of a sequence of functions commute? The following result provides a sufficient criteria.

Theorem 1. Let $(f_n)$ be a sequence function on an open interval $I$. Suppose

1. for each $n$, $f_n$ is $C^1$ on $I$.
2. the sequence $(f'_n)$ converges uniformly on $I$ and
3. for some $a \in I$, the real sequence $(f_n(a))$ converges. Then $\lim_n f_n$ exists, is differentiable on $I$ and

$$ (\lim_n f_n)' = \lim_n f_n'. $$

Proof. By the 1st from of the FTC, for any $x \in I$ and any $n \in \mathbb{N}$,

$$ \int_a^x f_n' = f_n(x) -f_n(a) $$

Let $g$ be the limit function of the $f_n'$s. By Theorem 6.2.4, $\lim_n (f_n(x)-f_n(a)) = \lim_n \int_a^x f_n' = \int_a^x g$ for any $x \in I$.

By assumption, $\lim_n f_n(a)$ exists and so does $\lim_n f_n(x)$ because $f_n(x) = (f_n(x)-f_n(a)) + f_n(a)$. Setting $f(x) = \lim_n f_n(x)$, then $$ f(x) - f(a) = \int_a^x g \qquad (x \in I). $$ Since $f_n'$s are continuous and $f_n' \to g$ uniformly, therefore $g$ is continuous (by 6.2.2) and so $f' = g$ by the 2nd form of the FTC.

Power Series

Let's apply what we have learnt to the studies of power series.

Suppose $(a_n)$ is a sequence of real number with $\limsup |a_n|^{1/n} \in [0,\infty)$.

Then the power series $\ds \sum_{k} a_k(x-x_0)^k$ has radius of convergence $R$ where

$$0 < R = \frac{1}{\limsup|a_n|^{1/n}} \le \infty$$

and converges (absolutely) pointwise on the open interval $(x_0 -R, x_0+R)$

So, the first natural question would be:

Must a power series, meaning its sequence of partial sums, converge uniformly on its open interval of convergence?

The answer is "no" as the following example shows

Example The power series $\ds \sum_{k=0}^{\infty} x^k$ converges to the function $f(x) = \dfrac{1}{1-x}$ on $(-1,1)$.

But for each $n \ge 1$, the difference between $f(x)$ and $\ds s_n(x)=1+x+ \cdots + x^{n-1}=\frac{1-x^{n}}{1-x}$, the $n$-th partial sum of the series, is

$$f(x) -s_n(x) = \frac{1}{1-x} - \frac{1-x^n}{1-x} = \frac{x^n}{1-x}$$

And that is clearly not bounded above on $(-1,1)$ as $\ds \lim_{x \to 1^{-}} \frac{x^n}{1-x} = +\infty$.

Thus, $\norm{f-s_n} = \sup\{|f(x)-s_n(x)| \colon x \in (-1,1)\} = +\infty$ for each $n$ and so the convergence of $s_n$ to $f$ is not uniform on $(-1,1)$.

That said, however

Theorem 2. A power series $\ds \sum_{k=0}^{\infty} a_k(x-x_0)^k$ with positive radius of convergence $R$ converges uniformly on $[x_0-R', x_0+R']$ for any $0 < R' < R$.

Proof. Pick any $0< R' < R$. Since

$$ \limsup |a_n|^{1/n} = \frac{1}{R} < \frac{1}{R'} $$

There exists $N \in \Nn$, such that $$u_N = \sup\{|a_k|^{1/k} \colon k \ge N\} < \frac{1}{R'}.$$

So for all $k \ge N$ and for any $x \in [x_0 -R', x_0 +R']$,

$$\left| a_k(x-x_0)^k \right| \le |a_k|(R')^k = (|a_k|^{1/k}R')^k \le (u_NR')^k $$

Let $$M_k = \begin{cases} |a_k|(R')^k & 0 \le k \le N-1 \\ (u_NR')^k & k \ge N \end{cases}$$

Then for all $k \ge N$, $|a_k(x-x_0)^k| \le M_k$ for all $x \in [x_0-R',x_0+R']$.

Since $0< u_NR' < 1$, the series $\sum M_k$ converges and hence $\sum a_k(x-x_0)^k$ converges uniformly on $[x_0-R',x_0+R']$ by Weierstrass's M-test.

Consequently, we now can prove Theorem 2 in Notes 02.

Proof. Let $\ds \sum_k a_k(x-x_0)^k$ be a power series with a positive radius of convergence $R$. Denote by $f(x)$ the function represented by the power series on $(x_0-R,x_0+R)$. For each $0< R' < R$, $$s_n(x):= \sum_{k=0}^n a_k(x-x_0)^k$$ is $C^1$ (since $s_n(x)$ is just a polynomial) on $(x_0-R',x_0+R')$. Since the power series $\ds \sum_k^\infty ka_k(x-x_0)^{k-1}$ also has radius of convergence $R$, therefore $$ s_n'(x) = \sum_{k=1}^n ka_k(x-x_0)^{k-1}$$ converges uniformly on $[x_0-R',x_0+R']$ (and so on $(x_0-R',x_0+R')$) by Theorem 2. Also, clearly we have $(s_n(x_0))$ is the constant sequence $(a_0)$ and hence convergent. Therefore, by Theorem 1, we conclude that $f(x) = \lim_n s_n(x)$ is differentiable on $(x_0-R',x_0+R')$ and $$ f'(x) = \lim_n s_n'(x) = \sum_{k=1}^{\infty} ka_{k}(x-x_0)^{k-1}$$

Since $0 < R' < R$ is arbitrary, we conclude that $f'(x) = \sum_{k=1}^{\infty} ka_k(x-x_0)^{k-1}$ on $(x_0 - R, x_0 + R)$.

Using Theorem 6.2.4 above, one can show that on $(x_0-R,x_0+R)$

$$ \int_{x_0}^x f(t)dt = \sum_{k=0}^{\infty} \frac{a_k}{k+1}(x-x_0)^{k+1} $$

We left the details as a exercise.