Conditional Convergence

In this post I wanted to discuss an unintuitive fact related to infinite series which often leads to confusion among early students of Real Analysis.

Even if you have not seen the formal definition of a series before you may have seen examples of them, such as the harmonic series \sum_{n=1}^{\infty} \frac{1}{n} or the infinite geometric series \sum_{n=0}^{\infty} a^n, perhaps calling them “infinite sums”.

The notation using the greek letter sigma can be quite misleading, since series do not always behave as nicely as one might initially expect. We will discuss what strange things can happen if one is not careful.

Formally, given an infinite sequence (a_n), a series is nothing but another special sequence (s_n) where s_n := a_0 + ... + a_n \;\;\;\forall n\in\mathbb{N}.
This sequence is denoted \sum_{n=0}^{\infty} a_n and we call the s_n partial sums of the series.

Apart from obvious issues, such as divergence to infinity (i.e the fact that the sum can grow uncontrollably large in either direction), there are other more subtle issues where series may behave differently from honest sums.

Consider a permutation f:\mathbb{N} \rightarrow \mathbb{N} and a convergent series \sum_{n=0}^{\infty} a_n.
We may ask: Does \sum_{n=0}^{\infty} a_n converge to the same value as \sum_{n=0}^{\infty} a_{f(n)}\;?
Or to give a more concrete informal example:
Is 1-\frac{1}{2} + \frac{1}{3} - \frac{1}{4} + \frac{1}{5} - \frac{1}{6} + \frac{1}{7} - ... +\frac{1}{n} - ... the same as (1 + \frac{1}{3}) - \frac{1}{2} + (\frac{1}{5} + \frac{1}{7}) - \frac{1}{4} + ... - \frac{1}{2n} + (\frac{1}{2n+3} + \frac{1}{2n+5}) - \frac{1}{2n+2} + ... \; ?

Perhaps surprisingly the answer is No in general.
The example I gave above was a rearrangement of the alternating harmonic series.
It is a well known fact that this series converges to ln 2 and one can show that the rearrangement converges to something different, namely \frac{3}{2}ln 2.
The diagram below shows the numeric value of the first 30 partial sums of either series.


It seems like our intuition about series as ordinary sums break down here.
In fact as we will show in the rest of this post, matters are even worse.
It turns out that you can rearrange this series (and other series like it) to converge to any real number you like! If you wish you can even make it diverge to infinity.

The defining property of series that converge in this way but are sensitive to rearrangement is called Conditional Convergence.
Usually this property is stated in another way, as a series which is convergent but not Absolutely Convergent.
A series \sum_{n=0}^{\infty} a_n is Absolutely Convergent if \sum_{n=0}^{\infty} |a_n| converges.
The true meaning of this property is somewhat hidden behind this definition.

As it happens,
convergent series which are insensitive to rearrangement are precisely those which are absolutely convergent.

Let’s prove this statement in one direction:

Proposition 1:

Let \sum_{n=0}^{\infty} a_n be an absolutely convergent series, converging to L.

Then \sum_{n=0}^{\infty} a_{f(n)} also converges to L for every permutation f:\mathbb{N} \rightarrow \mathbb{N}.

Informal Proof:
Let s_n = \sum_{k=0}^{n} a_k and r_n = \sum_{k=0}^{n} a_{f(k)}.

To show convergence our goal is to show that the quantity |r_n - L| can be made arbitrarily small for sufficiently large n, that is to say |r_n - L| \rightarrow 0 as n \rightarrow \infty.

Since (s_n) converges to L we know that |s_n - L| \rightarrow 0 as n \rightarrow \infty.
To understand if (r_n) also converges to L we should suggestively compare (r_n) with (s_n). For if (r_n) and (s_n) get very close to each other, knowing that (s_n) gets very close to L we ought to see that (r_n) gets very close to L as well for sufficiently large n.

This type of idea in analysis is usually expressed via the triangle inequality:

|r_n - L| = |r_n - L + s_n - s_n| \leq |r_n - s_n| + |s_n - L|\;\;\;(*)

We know how to manage the quantity |s_n - L|.
We want to know what happens to |r_n - s_n| as n grows large.

Consider a fixed N \in \mathbb{N}.
Since f:\mathbb{N} \rightarrow \mathbb{N} is a bijection (permutations are by definition bijections) \exists n_0,...,n_N \in \mathbb{N} s.t f(n_0) = 0, ..., f(n_N) = N.

Let M := max\{n_0,...,n_N\} \geq N and consider the set of subscripts D := \{1,...,M\} \setminus \{n_0,...,n_N\} that do not belong to \{1,...,N\} after applying f.

|r_M - s_N| = |\sum_{k=0}^M a_{f(k)} - \sum_{k=0}^N a_{f(n_k)} | = |\sum_{k \in D} a_{f(k)}| \leq^{(\bigtriangleup-ineq)} \sum_{k \in D} |a_{f(k)}| \leq^{(more\;positive\;terms)} \sum_{k=N+1}^{\infty} |a_k|

This is where the hypothesis about absolute convergence kicks in.
Absolute convergence implies \sum_{k=N+1}^{\infty} |a_k| \rightarrow 0 as N \rightarrow \infty.
Thus since M \rightarrow \infty \; as \; N \rightarrow \infty we find that |r_n - s_n| \rightarrow 0 as n \rightarrow \infty.

In combination with |s_n - L| \rightarrow 0 as n \rightarrow \infty we get from (*) that |r_n - L| \rightarrow 0 as n \rightarrow \infty.

This shows that \sum_{n=0}^{\infty} a_{f(n)} converges to L.

To prove the converse we need to show that all convergent series that are not absolutely convergent (i.e conditionally convergent) are non-invariant under rearrangement.

From now on we will assume \sum a_n is conditionally convergent.

As I hinted initially something stronger holds true for conditionally convergent series.
Not only are the limits of these series sensitive to rearrangement, these series can be rearranged to converge to any real number. This result is commonly known as Riemann’s rearrangement theorem.

The idea behind the proof is essentially to construct a rearrangement by indefinitely adding positive and negative terms from the original series in order to oscillate around the chosen real value, and show that these oscillations get smaller and smaller.

Define a_n^+ = \begin{cases} a_n \;\; if\;\; a_n > 0 \\ 0 \;\;\;\;otherwise \end{cases} and a_n^- = \begin{cases} a_n \;\; if\;\; a_n < 0 \\ 0 \;\;\;\;otherwise \end{cases}

There are a few technical issues that need to be resolved before we can claim the construction idea is well defined.

  1. Are there infinitely many non-zero a_n^+, a_n^- so we can indefinitely add positive and negative terms?
  2. Is \sum a_n^+ unbounded above and \sum a_n^- unbounded below so we have a chance at getting close to any real value?
  3. Does a_n^+, a_n^- both converge to 0 so that we can make oscillations arbitrarily small?

Answer to all 3 questions need to be Yes for this construction to work.

To see (1) we could argue by contradiction and e.g assume there are only finitely many non-zero a_n^- (the a_n^+ case works similarly). In fact let’s assume something even weaker and just suppose that \sum a_n^- is convergent.

Then \sum |a_n| = \sum (a_{n}^{+} - a_{n}^{-})=\sum (a_{n}^{+} + a_{n}^{-})-2\sum a_n^-=\sum a_n -2\sum a_n^-

By assumption both series on the right hand side converge, implying \sum |a_n| is convergent. But we assumed \sum a_n was not absolutely convergent so this is a contradiction. Therefore \sum a_n^- cannot converge and in particular this means the number of negative terms are infinite.

From this we can also deduce (2) because \sum a_n^- is monotonically decreasing (being a series of non-positive terms) and if it was bounded below then a  standard result from real analysis (Monotone Convergence Theorem) says that \sum a_n^- needs to converge which we know it cannot. Hence \sum a_n^- cannot be bounded below.

A symmetrical argument establishes (1) and (2) for \sum a_n^+.

(3) follows directly from the fact that terms of convergent series necessarily tend to zero.
To see this just write a_n = \sum_{k=0}^n a_k -\sum_{k=0}^{n-1} a_k and notice that the right hand side converges to zero as n \rightarrow \infty since both series converge to the same value.

Hence we can answer all 3 questions in the affirmative.

Now let’s sketch a proof of Riemann’s Rearrangement Theorem


Riemann Rearrangement Theorem:
Let \sum_{k=0}^{\infty} a_k be a conditionally convergent series and let L \in \mathbb{R}.

Then there exists a permutation f:\mathbb{N} \rightarrow \mathbb{N} s.t \sum_{k=0}^{\infty} a_{f(k)} converges to L.

Proof (Sketch):
The proof is constructive (the best kind).

Let L \in \mathbb{R}.

By (2):

since \sum_{k=0}^{\infty} a_k^+ is unbounded above we can choose the smallest n_1 \in \mathbb{N} s.t
a_0^+...+a_{n_1}^+ > L.

since \sum_{k=0}^{\infty} a_k^- is unbounded below we can choose the smallest m_1 \in \mathbb{N} s.t
a_0^+...+a_{n_1}^++a_0^-...+a_{m_1}^- < L.

We can continue to choose the smallest n_{2}\; \textgreater \;n_{1} s.t a_0^+...+a_{n_1}^++a_0^-...+a_{m_1}^-+a_{n_1+1}^+...+a_{n_2}^+ \; \textgreater \; L etc..

Inductively if we denote P_i := a_{n_{i-1}+1}^+...+a_{n_i}^+ and N_i := a_{m_{i-1}+1}^-...+a_{m_i}^-
we obtain a rearrangement P_1 + N_1 + ... + P_k + N_k + ... of \sum_{k=0}^{\infty} a_k s.t
P_1 + N_1 + ... + P_k \; \textgreater \; L and P_1 + N_1 + ... + P_k + N_k \; \textless \; L \forall k \in \mathbb{N}.

We can make this definition since we have an infinite supply of positive and negative terms by (1).

An arbitrary partial sum in the rearrangement has one of below two forms:
s_{n} = P_1 + N_1 + ... + P_k + N_k + a_{n_k+1}^+ + ... + a_{n_k+r}^+ or
s_{n} = P_1 + N_1 + ... + P_k + a_{m_{k-1}+1}^- + ... + a_{m_{k-1}+r}^-.

We want to show |s_n - L| \rightarrow 0 as n \rightarrow \infty.

We clearly have P_1+N_1+...+P_k+N_k - L \leq s_n - L \leq P_1+N_1+...+P_k - L.

By construction,
P_1+N_1+...+P_k+N_k - a_{m_k}^- \geq L and
P_1+N_1+...+P_k - a_{n_k}^+ \leq L.

Therefore a_{m_k}^- \leq s_n - L \leq a_{n_k}^+

By (3) we know that a_{n_k}^+, a_{m_k}^- \rightarrow 0 as k \rightarrow \infty so s_n -L is being squeezed between two sequences converging to zero and is therefore forced to converge to zero as well by the so called Sandwich Theorem.

This shows that the rearranged series converges to L.
Since L was arbitrary the result follows.


To put back some intuition behind this phenomenon the proof tells us that conditionally convergent series are a result of two divergent series \sum a_k^+, \sum a_k^- cancelling out each other to allow for convergence.
In the cancelling race between the two series we can give one series an “infinite head start” by pushing terms from the other series further away into infinity. This way we can accumulate terms quicker in the partial sums from one of the series, at a pace we set, and thus have it converge to anything we like. I am being purposely vague here, but hopefully this makes some intuitive sense.

In a way one could think of the terms in a conditionally convergent series as being infinite streams of positive revenue and negative expenditure. By rearranging expenditures ahead in time we could alter our book balance (and in theory make infinite profit).
Funny enough this is the basic idea behind Ponzi schemes.
Of course in reality Ponzi schemes do not generate infinite profit.
It inevitably falls apart because the criminal cannot find sufficiently many revenue sources to sustain the expenditure commitments of the scheme, so when the scheme finally starts to collapse the criminal has to run away with the money.
The basic fallacy in applying this idea to real life is that things are usually not infinite even though people sometimes behave as if they are. Economic bubbles are a good real world example which behave very similar to Ponzi schemes but on a much larger scale. Eventually money is solely being made at expense of new investors entering the market, the profit being completely detached from the markets intrinsic value. Of course the number of investors nor their money is infinite, so sooner or later people are bound to lose money as the bubble implodes due to lack of new buying market participants (i.e there are only sellers left).

Exercise 1( Alternating Harmonic Series – Intermediate):
Prove that \sum_{n=1}^{\infty} (-1)^{n+1}\frac{1}{n} = ln(2).

(Hint: Look at the partial sums s_{2n} and express it as a Riemann sum like in the previous post YAPP)


Exercise 2( Conditional Divergence – Easy):
Prove that every conditionally convergent series can be rearranged to diverge to infinity.


Exercise 3( Huh? – Easy):

What is the fallacy in below argument?

0 = 0 + 0 + 0 + 0 + .... =
(1-1) + (1-1) + (1-1) + (1-1) + .... =
1 + (-1+1) + (-1+1) + (-1+1) + (-1+1) + .... =
1 + 0 + 0 + 0 + 0 + .... = 1

This entry was posted in Analysis, Uncategorized and tagged , , , , , , , , , , , . Bookmark the permalink.