Laws of Large Numbers
Y. Eddie Lu, Summer 2026
ECON 8002 course index · Lecture 7 of 9
Orientation
Laws of large numbers
Why can sample averages estimate population objects, and exactly what dependence and tail assumptions make that statement true?
How to use this page
For every LLN, name the average, its centering target, the convergence mode, and the assumption that shrinks dispersion.
How should I think about Lecture 7?
Independence lets variances add.
Integrability makes the population target meaningful.
\(\longrightarrow\)
An average suppresses idiosyncratic noise and reveals a population mean.
\(\longrightarrow\)
WLLN gives consistency in probability.
SLLN gives pathwise long-run regularity.
Course map
- Lectures 1–4: probability and conditioning
- Lecture 5: convergence modes
- Lecture 6: transformations
- Lecture 7: laws of large numbers (current lecture)
- Lectures 8–9: weak limits, central limit theorems, and inference
Prerequisite retrieval
You need \(X_n\to_pX\), \(X_n\to X\) a.s., independence, \(\operatorname{Var}(\sum_iX_i)\), Chebyshev’s inequality, and Slutsky’s theorem. Define \(S_n=\sum_{i=1}^nX_i\).
What an LLN claims
Concrete opening: default frequency
If \(X_i\in\{0,1\}\) records default on loan \(i\), then
\[\bar X_n=\frac1n\sum_{i=1}^nX_i\]
is the empirical default rate. The LLN says when it approaches a population probability.
The average, not the observation
The LLN is not \(X_n\to E[X_1]\). It concerns
\[\bar X_n=\frac{S_n}{n}.\]
Individual returns or defaults can remain highly random forever while their average stabilizes.
Retrieval check.
What random object is claimed to converge in an IID LLN?
The sample average \(n^{-1}\sum_{i=1}^nX_i\), not the final observation \(X_n\).
Strong and weak laws
\[\bar X_n\xrightarrow{\mathrm{a.s.}}\mu\quad\text{(SLLN)},\qquad \bar X_n\xrightarrow{p}\mu\quad\text{(WLLN)}.\]
SLLN implies WLLN and is a stronger pathwise statement. In the source’s exact IID results, the truncated-centering WLLN is more permissive than the ordinary SLLN; sufficient conditions need not always differ in other formulations.
Kronecker’s lemma
If \(0<a_n\uparrow\infty\) and \(\sum_{n\ge1}x_n/a_n\) converges, then
\[\frac1{a_n}\sum_{i=1}^nx_i\to0.\]
It converts a controlled weighted series into a vanishing average. Treat it as a bridge in SLLN proofs.
IID laws of large numbers
IID WLLN: source’s exact form
For IID \(X_i\), there exists a centering sequence \(\tau_n\) with
\[\frac1n\sum_{i=1}^n(X_i-\tau_n)\xrightarrow{p}0\]
iff \(nP(|X_1|>n)\to0\), where
\[\tau_n=E[X_1\mathbf1_{\{|X_1|\le n\}}].\]
Why truncated centering?
The exact WLLN permits some heavy tails. When \(E|X_1|<\infty\), truncation becomes asymptotically harmless and \(\tau_n\to E[X_1]\).
Retrieval check.
Why should we not automatically replace \(\tau_n\) by \(E[X_1]\) in the source’s WLLN?
The theorem permits cases in which the ordinary mean may not be available; its target is explicitly truncated.
IID SLLN
For IID \(X_i\),
\[\bar X_n\xrightarrow{\mathrm{a.s.}}\mu\ \text{for some finite }\mu \quad\Longleftrightarrow\quad E|X_1|<\infty,\]
and then \(\mu=E[X_1]\). This is an exact iff result for the IID case.
Weighted IID consequence
If \(E|X_1|<\infty\) and deterministic weights \(|a_i|\le C\), then
\[\frac1n\sum_{i=1}^na_i(X_i-E[X_1])\xrightarrow{\mathrm{a.s.}}0.\]
Bounded weights cannot amplify an individual shock without limit.
Bernoulli frequency
For IID \(X_i\sim\operatorname{Bernoulli}(p)\),
\[\bar X_n\xrightarrow{\mathrm{a.s.}}p.\]
Because \(E|X_1|=p<\infty\). This is the frequentist frequency interpretation, not a definition of probability.
Retrieval check.
Does this result say the next draw becomes equal to \(p\)?
No. Each \(X_i\) remains binary; only the running frequency converges.
Independent heterogeneous sequences
Independent but heterogeneous observations
When distributions differ, the natural centered average is
\[\frac1n\sum_{i=1}^n(X_i-E[X_i]).\]
Do not write a common \(\mu\) unless the average of the means has a relevant limit.
Independent-sequence SLLN
If independent \(X_i\) satisfy \(E|X_i|<\infty\) and, for some \(\alpha\in[1,2]\),
\[\sum_{i=1}^\infty i^{-\alpha}E|X_i|^\alpha<\infty,\]
then \(n^{-1}\sum_{i=1}^n(X_i-E[X_i])\to0\) a.s.
Independent-sequence WLLN
If independent \(X_i\) satisfy
\[\sup_i\operatorname{Var}(X_i)\le C<\infty,\]
then
\[\frac1n\sum_{i=1}^n(X_i-E[X_i])\xrightarrow{p}0.\]
This is a sufficient condition, not a necessary one.
Retrieval check.
What centering appears in the non-IID WLLN?
\(n^{-1}\sum_{i=1}^nE[X_i]\), not automatically \(E[X_1]\).
Proof strategy: make dispersion vanish
Set \(S_n=\sum_{i=1}^nX_i\). Chebyshev reduces the task to showing
\[\operatorname{Var}(S_n)/n^2\to0.\]
Independence and uniformly bounded variances make \(\operatorname{Var}(S_n)=O(n)\).
Proof: Chebyshev
For every \(\varepsilon>0\),
\[P\left(\left|\frac{S_n-E[S_n]}n\right|>\varepsilon\right) \le\frac{\operatorname{Var}(S_n)}{\varepsilon^2n^2}.\]
Proof: where independence enters
\[\operatorname{Var}(S_n)=\sum_{i=1}^n\operatorname{Var}(X_i) \le nC,\]
because covariances vanish under independence. Therefore the preceding probability is at most \(C/(\varepsilon^2n)\to0\).
Retrieval check.
Which equality fails if the observations are correlated?
\(\operatorname{Var}(S_n)=\sum_i\operatorname{Var}(X_i)\); covariance terms remain.
Nonexample: perfect dependence
Let \(X_i=Z\) for every \(i\), with integrable, nondegenerate \(Z\), such as \(Z\sim N(0,1)\). Then \(\bar X_n=Z\) for every \(n\), so it does not converge to \(E[Z]\) in probability. A large sample alone does not diversify common shocks.
Nonexample: no finite target
For IID Pareto \(X_i\) with tail index at most \(1\), \(E|X_1|=\infty\). The ordinary SLLN conclusion \(\bar X_n\to E[X_1]\) cannot even be stated with a finite target.
LLN versus CLT
\[\bar X_n-\mu=o_p(1)\]
is consistency. A CLT instead describes the nonvanishing normalized fluctuation \(\sqrt n(\bar X_n-\mu)\). LLN gives no root-\(n\) rate on its own.
Retrieval check.
Does a WLLN by itself imply \(\sqrt n(\bar X_n-\mu)=O_p(1)\)?
No. That is a rate claim requiring stronger information, often a CLT.
Rates and the CLT connection
Example 20: CLT-scaled averages
Suppose independent \(X_i\) satisfy
\[\frac{\sum_{i=1}^n(X_i-E[X_i])}{\sigma_n}\xrightarrow{d}N(0,1), \qquad \sigma_n^2=\operatorname{Var}(S_n).\]
Then the centered average converges in probability to zero iff \(\sigma_n/n\to0\).
Sufficiency: use Slutsky
If \(\sigma_n/n\to0\), then
\[\frac{S_n-E[S_n]}n= \frac{\sigma_n}{n}\frac{S_n-E[S_n]}{\sigma_n}\xrightarrow{d}0.\]
Since the limit is constant, it also converges in probability to zero.
Necessity: the source needs a subsequence repair
The source says “without loss of generality” that \(\sigma_n/n\to\psi\). That does not follow from failure to converge to zero. Correct approach: if \(\sigma_n/n\not\to0\), choose a subsequence \(n_k\) with \(\sigma_{n_k}/n_k\ge\varepsilon>0\).
Retrieval check.
What compactness step is needed after selecting \(n_k\)?
Extract a further subsequence on which \(\sigma_{n_k}/n_k\to\psi\in[\varepsilon,\infty]\) in the extended nonnegative line.
Necessity: finite positive subsequential limit
If \(\sigma_{n_k}/n_k\to\psi\in(0,\infty)\), Slutsky yields
\[\frac{S_{n_k}-E[S_{n_k}]}{n_k}\xrightarrow{d}\psi Z, \qquad Z\sim N(0,1),\]
which cannot converge in probability to zero.
Necessity: infinite subsequential limit
If \(\sigma_{n_k}/n_k\to\infty\), choose \(M>0\) with \(P(|Z|>2M)>0\). By c.d.f. convergence, the standardized sums exceed \(M\) in magnitude with probability bounded away from zero eventually; multiplying by the diverging scale rules out convergence in probability to zero.
Why this repair matters
A global sequence need not converge merely because it fails to converge to zero. Asymptotic contradictions are often proved on a carefully chosen subsequence.
Applications, synthesis, and retrieval
Econometric connection: sample moments
If \(W_i\) are IID and \(E\|g(W_i,\theta_0)\|<\infty\), then
\[\frac1n\sum_{i=1}^ng(W_i,\theta_0)\xrightarrow{p}E[g(W,\theta_0)]\]
is the first consistency step for method of moments, GMM, and many extremum estimators.
Dependent-data versions replace IID sampling with an appropriate stationary, ergodic, mixing, or other weak-dependence LLN.
Finance connection: diversification needs assumptions
An equally weighted portfolio average becomes stable only if dependence is weak enough for its variance to shrink. Perfect common shocks are the counterexample.
Retrieval check.
In the Chebyshev proof, what order must \(\operatorname{Var}(S_n)\) have relative to \(n^2\)?
It must be \(o(n^2)\). Uniformly bounded individual variances under independence give the stronger \(O(n)\) bound.
Assumption audit
| claim | assumption doing the work |
|---|---|
| IID SLLN | IID and \(E|X_1|<\infty\) |
| heterogeneous WLLN | independence and \(\sup_i\operatorname{Var}(X_i)<\infty\) |
| Chebyshev proof | variance additivity |
| CLT-to-LLN scale result | CLT plus \(\sigma_n/n\to0\) |
| necessity repair | subsequence extraction |
Common errors
- Do not claim \(X_n\to E[X_1]\).
- Do not replace non-IID centering by a fixed mean without justification.
- Bounded variance is sufficient, not necessary.
- A CLT alone is not an LLN.
- \(\sigma_n\) is deterministic: it is a population standard deviation.
LLN proof blueprint
- State the average and its centering.
- Choose the convergence mode.
- Bound its deviation probability, often by Chebyshev.
- Compute the variance, naming the independence step.
- Check its order relative to \(n^2\).
- For a CLT argument, separate the normalized random term from its deterministic scale.
Exit ticket 1
State the finite-mean IID SLLN in one line, including its mode.
Exit ticket 2
Prove the independent bounded-variance WLLN using Chebyshev.
Exit ticket 3
Explain why the subsequence is needed in Example 20’s necessity argument.
Mastery checklist
- I can distinguish WLLN and SLLN.
- I can state the source’s exact IID WLLN and SLLN.
- I can prove the bounded-variance independent WLLN.
- I can identify when dependence breaks variance additivity.
- I can connect LLN consistency, CLT scaling, and the corrected subsequence argument.