Properties of Probability Distributions
Y. Eddie Lu, Summer 2026
ECON 8002 course index · Lecture 3 of 9
Orientation
How to use this page
When you see a distribution, ask four questions: relative to which measure is its density defined; which moments exist; which inequality controls its tails; and which transform identifies it?
How should I think about this lecture?
Densities and transformations
Means and covariance
\(\longrightarrow\)
Describe distributions in ways that support estimation and limit theory
\(\longrightarrow\)
Tail inequalities
MGFs and characteristic functions
Course map
- Measure spaces
- Integration and Radon–Nikodym derivatives
- Distribution properties (current lecture)
- Conditional expectation
- Convergence modes
- Continuous mapping and Slutsky
- Laws of large numbers
- Weak limits and the delta method
- Central limit theorems and inference
Prerequisite retrieval
\(E[g(X)]=\int g(x)dP_X(x)\), a density is an RN derivative relative to a reference measure, and a joint law is needed to study dependence.
Retrieval check.
What reference measure underlies an ordinary continuous PDF?
Lebesgue measure \(\lambda\), so \(p=dP_X/d\lambda\).
Densities and transformations
Multivariate normal distribution
For \(X\in\mathbb R^d\), \(X\sim N_d(\mu,\Sigma)\) with \(\mu\in\mathbb R^d\) and symmetric positive-definite \(\Sigma\in\mathbb R^{d\times d}\) has density \[p(x)=(2\pi)^{-d/2}\det(\Sigma)^{-1/2}\exp\left\{-\tfrac12(x-\mu)^\top\Sigma^{-1}(x-\mu)\right\}.\]
Geometry of the normal density
The quadratic form \((x-\mu)^\top\Sigma^{-1}(x-\mu)\) is squared Mahalanobis distance. Its level sets are ellipsoids: covariance changes their scale and orientation.
The displayed density requires \(\Sigma\succ0\). If \(\Sigma\) is singular, the distribution may exist but has no \(d\)-dimensional Lebesgue density.
A mixed distribution: censoring at a cap
Let \(Y=\min\{c,X\}\) where \(X\) has continuous CDF \(F_X\) and \(P(X>c)>0\). Then \(Y\) has a continuous part below \(c\) and an atom \[P(Y=c)=P(X\ge c).\] It is neither purely discrete nor absolutely continuous with respect to Lebesgue measure.
Dominating measure for a mixed law
Use \(m=\lambda+\delta_c\). Then \(P_Y\ll m\), and \(dP_Y/dm\) has a continuous value below \(c\) plus an atomic value at \(c\). “No ordinary PDF” does not mean “no density relative to any measure.”
Retrieval check.
Where does the atom of \(Y=\min\{c,X\}\) come from?
Every realization \(X\ge c\) maps to the single value \(c\), concentrating positive probability there.
Non-injective transformations: \(Y=X^2\)
Assume \(X\) has a Lebesgue density, hence a continuous CDF. For \(y\ge0\), \[F_Y(y)=P(-\sqrt y\le X\le\sqrt y)=F_X(\sqrt y)-F_X(-\sqrt y).\] If \(X\) has Lebesgue density \(p_X\), then for \(y>0\), \[p_Y(y)=\frac{p_X(\sqrt y)+p_X(-\sqrt y)}{2\sqrt y}.\]
Boundary case: squared normal
If \(Z\sim N(0,1)\), then \(Z^2\sim\chi_1^2\) with \[p_{Z^2}(y)=\frac{1}{\sqrt{2\pi y}}e^{-y/2}\mathbf1\{y>0\}.\] Both roots matter. A one-branch Jacobian answer is wrong.
Retrieval check.
Why does the density formula for \(X^2\) include two terms?
Both \(\sqrt y\) and \(-\sqrt y\) map to \(y\); probabilities from both preimage branches must be added.
Moments and dependence
Moments: definitions
For a real random variable \(X\) and \(r>0\), the \(r\)th absolute moment is \(E|X|^r\) when finite. Integer raw moments are \(E[X^k]\); if \(\mu=E[X]\) exists, central moments are \(E[(X-\mu)^k]\).
Higher moments buy lower moments
If \(E|X|^r<\infty\) and \(0<s<r\), then \(E|X|^s<\infty\). The converse fails: a Pareto-type tail can have finite mean but infinite variance.
Variance, covariance, correlation
Assume the relevant variables have finite second moments. Then \[\operatorname{Var}(X)=E[(X-E[X])^2]=E[X^2]-(E[X])^2,\] \[\operatorname{Cov}(X,Y)=E[(X-E[X])(Y-E[Y])],\qquad \operatorname{Corr}(X,Y)=\frac{\operatorname{Cov}(X,Y)}{\sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)}}.\] Correlation requires positive finite variances.
Retrieval check.
Does \(E|X|<\infty\) guarantee \(\operatorname{Var}(X)<\infty\)?
No. Variance requires a finite second moment.
Random vectors and portfolio variance
For \(X\in\mathbb R^d\) with mean \(\mu\) and covariance matrix \(\Sigma\), and fixed \(a\in\mathbb R^d\), \[E[a^\top X]=a^\top\mu,\qquad \operatorname{Var}(a^\top X)=a^\top\Sigma a.\] For returns \(R\), this is the mean and variance of portfolio return \(w^\top R\).
Proof map: quadratic-form variance
- Center: \(a^\top X-a^\top\mu=a^\top(X-\mu)\).
- Square the scalar: \((a^\top(X-\mu))^2=(X-\mu)^\top aa^\top(X-\mu)\).
- Take expectation and use linearity.
- Recognize \(E[(X-\mu)(X-\mu)^\top]=\Sigma\) to obtain \(a^\top\Sigma a\).
Covariance bound and correlation range
Cauchy–Schwarz gives \[\operatorname{Cov}(X,Y)^2\le\operatorname{Var}(X)\operatorname{Var}(Y).\] Hence correlation lies in \([-1,1]\) when defined. The statement concerns population quantities, not a sample correlation coefficient.
Retrieval check.
If \(w=(1/2,1/2)^\top\) and \(\Sigma=\begin{pmatrix}4&1\\1&9\end{pmatrix}\), compute \(w^\top\Sigma w\).
\((1/4)(4+2+9)=15/4\).
Product expectations under independence
For independent \(X_1,\ldots,X_k\) with each \(X_j\in L^1\), \[E[X_1\cdots X_k]=\prod_{j=1}^kE[X_j].\] Independence plus the individual \(L^1\) conditions implies \(E|X_1\cdots X_k|=\prod_jE|X_j|<\infty\).
Moment inequalities
Why moment inequalities matter
Inequalities turn moment information into bounds on probabilities or on other moments. They are the machinery behind consistency proofs and uniform-integrability arguments.
Hölder, Lyapunov, and Minkowski
For conjugate \(p,q>1\) with \(p^{-1}+q^{-1}=1\), \(X\in L^p\), and \(Y\in L^q\), \[E|XY|\le(E|X|^p)^{1/p}(E|Y|^q)^{1/q}.\] For \(0<r\le s\) on a probability space and \(X\in L^s\), \(\|X\|_r\le\|X\|_s\) (Lyapunov). For \(p\ge1\) and \(X,Y\in L^p\), \[\|X+Y\|_p\le\|X\|_p+\|Y\|_p\] (Minkowski). Cauchy–Schwarz is Hölder with \(p=q=2\).
Jensen’s inequality
If \(\varphi\) is convex on a convex domain \(D\), \(P(X\in D)=1\), \(X\) is integrable, and \(E[\varphi(X)]\) exists, then \[\varphi(E[X])\le E[\varphi(X)].\] For concave \(\varphi\), reverse the inequality. Thus, if \(W>0\) almost surely, \(0<EW<\infty\), and \(E|\log W|<\infty\), \[E[\log W]\le\log E[W].\]
Retrieval check.
For positive nondegenerate \(X\) with \(0<E[X]<\infty\) and \(E[X^{-1}]<\infty\), compare \(E[X^{-1}]\) and \((E[X])^{-1}\).
\(E[X^{-1}]\ge( E[X])^{-1}\) because \(x\mapsto1/x\) is convex on \((0,\infty)\).
Markov and Chebyshev
For nonnegative \(Z\) and \(a>0\), \(P(Z\ge a)\le E[Z]/a\). Applying this to \(Z=(X-E[X])^2\) yields \[P(|X-E[X]|\ge t)\le\frac{\operatorname{Var}(X)}{t^2}.\] Chebyshev is a distribution-free tail bound, not a normal approximation.
Standardized Chebyshev example
If \(Z=(X-\mu)/\sigma\) has mean \(0\) and variance \(1\), then \[P(|Z|\ge2)\le\frac14.\] It bounds a tail even without knowing the distribution of \(Z\).
Corollary 3.5: information inequality
Let \(f,g\ge0\) be integrable on a \(\sigma\)-finite measure space, assume \(g>0\) wherever \(f>0\), and suppose \[\int f\,d\mu\ge\int g\,d\mu>0.\] Then, whenever the left side is defined, \[\int f\log\left(\frac f g\right)d\mu\ge0.\] For probability densities both integrals equal \(1\); this is Gibbs’ inequality for Kullback–Leibler divergence, with equality iff \(f=g\) \(\mu\)-a.e.
Retrieval check.
Which nonnegative random variable do you feed to Markov to derive Chebyshev?
\((X-E[X])^2\), with threshold \(t^2\).
Transforms that identify laws
MGF and characteristic function
For \(X\in\mathbb R^d\) and \(t\in\mathbb R^d\), \[M_X(t)=E[e^{t^\top X}],\qquad \phi_X(t)=E[e^{it^\top X}].\] The CF always exists because \(|e^{it^\top X}|=1\); an MGF may be infinite.
If the transform is extended to complex arguments by \(M_X(z)=E[e^{z^\top X}]\) wherever this expectation exists, then \[\phi_X(t)=M_X(it).\]
Moments from derivatives
Under conditions allowing differentiation under the expectation, \[M_X^{(k)}(0)=E[X^k]\] in the scalar case. For vectors, mixed partial derivatives recover mixed moments. This is why the source calls it a moment-generating function.
Affine transformations
If \(Y=AX+b\), then \[M_Y(t)=e^{b^\top t}M_X(A^\top t),\qquad \phi_Y(t)=e^{ib^\top t}\phi_X(A^\top t).\] Check dimensions: \(A\) maps the dimension of \(X\) to that of \(Y\).
Retrieval check.
If \(M_X'(0)=3\) and \(M_X''(0)=13\), find \(E[X]\) and \(\operatorname{Var}(X)\).
\(E[X]=3\) and \(\operatorname{Var}(X)=13-3^2=4\).
Normal MGF
For \(X\sim N(\mu,\sigma^2)\), \[M_X(t)=\exp\{\mu t+\sigma^2t^2/2\}.\] Differentiating at zero gives \(E[X]=\mu\) and \(E[X^2]=\mu^2+\sigma^2\).
Uniqueness: transforms identify laws
If \(\phi_X(t)=\phi_Y(t)\) for all \(t\in\mathbb R^d\), then \(P_X=P_Y\). If MGFs agree and are finite on a neighborhood of \(0\), then the laws also agree.
Do not replace this with “all moments identify a distribution.” That claim needs extra conditions and can fail.
Real characteristic functions and symmetry
If \(X\) is symmetric about \(0\), then \(\phi_X(t)=E[\cos(t^\top X)]\) is real. Conversely, if \(\phi_X\) is real-valued for every \(t\), then \[\phi_{-X}(t)=\phi_X(-t)=\overline{\phi_X(t)}=\phi_X(t),\] so CF uniqueness gives \(X\overset d=-X\).
Independence turns sums into products
For independent random vectors \(X,Y\), \[\phi_{X+Y}(t)=\phi_X(t)\phi_Y(t),\qquad M_{X+Y}(t)=M_X(t)M_Y(t),\] where the MGF equation holds where expectations are finite.
Proof map: CF factorization
- Start with \(\phi_{X+Y}(t)=E[e^{it^\top X}e^{it^\top Y}]\).
- Independence makes the two complex-valued factors independent.
- Factor expectation: \(E[UV]=E[U]E[V]\).
- Recognize \(\phi_X(t)\phi_Y(t)\).
Connection to later CLTs
Sums of independent observations have CFs equal to products. After suitable centering and scaling, analyzing the limit of those products yields one standard route to the CLT.
Synthesis and retrieval
Assumption audit
| Assumption | What it buys |
|---|---|
| \(\Sigma\succ0\) | ordinary multivariate normal density |
| \(E|X|^r<\infty\) | \(r\)th absolute moment and lower moments |
| independence plus integrability | expectation/transform factorization |
| convexity and domain | Jensen direction |
| MGF finite near \(0\) | MGF uniqueness and derivatives |
Common errors and false converses
Uncorrelated does not imply independent. A density must name its reference measure. A transformation may have multiple inverse branches. Chebyshev is not a normal approximation. A CF is complex-valued even for real \(X\).
Calculation blueprint
- For transforms, find the complete preimage. 2. For moments, verify finiteness. 3. For a portfolio, write the quadratic form. 4. For an inequality, state curvature or nonnegativity. 5. For an MGF/CF, check existence and independence before factoring.
Exit ticket 1
Retrieval check.
Why can \(\min\{c,X\}\) fail to have a Lebesgue PDF?
It has an atom at \(c\), while any Lebesgue-density distribution assigns zero probability to singletons.
Exit ticket 2
Retrieval check.
State and prove the variance of \(a^\top X\).
\(\operatorname{Var}(a^\top X)=a^\top\Sigma a\) by centering, expanding the scalar square, taking expectations, and using the covariance matrix definition.
Exit ticket 3
Retrieval check.
What theorem shows that equal CFs imply equal distributions?
The characteristic-function uniqueness theorem, requiring equality for every \(t\in\mathbb R^d\).
Mastery checklist
You should now be able to classify density types, transform a non-injective random variable, compute population moments and portfolio variance, reject false independence converses, use Jensen/Markov/Chebyshev correctly, and state how CFs identify laws.
Built from ECON 8002 Lecture 3. The source uses overloaded Greek symbols; this page standardizes them as \(\mu,\Sigma,\lambda\) and makes moment-existence assumptions explicit.