Local limit theorem

Lindeberg–Lévy CLT. Suppose {\textstyle \{X_{1},\ldots ,X_{n}\}}  is a sequence of i.i.d. random variables with {\textstyle \mathbb {E} [X_{i}]=\mu }  and {\textstyle \operatorname {Var} [X_{i}]=\sigma ^{2}<\infty } . Then as {\textstyle n}  approaches infinity, the random variables {\textstyle {\sqrt {n}}({\bar {X}}_{n}-\mu )}  converge in distribution to a normal {\textstyle {\mathcal {N}}(0,\sigma ^{2})} :^[4]

{\displaystyle {\sqrt {n}}\left({\bar {X}}_{n}-\mu \right)\ \xrightarrow {d} \ {\mathcal {N}}\left(0,\sigma ^{2}\right).}

In the case {\textstyle \sigma >0} , convergence in distribution means that the cumulative distribution functionsof {\textstyle {\sqrt {n}}({\bar {X}}_{n}-\mu )}  converge pointwise to the cdf of the {\textstyle {\mathcal {N}}(0,\sigma ^{2})}  distribution: for every realnumber {\textstyle z} ,

{\displaystyle \lim _{n\to \infty }\mathbb {P} \left[{\sqrt {n}}({\bar {X}}_{n}-\mu )\leq z\right]=\lim _{n\to \infty }\mathbb {P} \left[{\frac {{\sqrt {n}}({\bar {X}}_{n}-\mu )}{\sigma }}\leq {\frac {z}{\sigma }}\right]=\Phi \left({\frac {z}{\sigma }}\right),}

where {\textstyle \Phi (z)}  is the standard normal cdf evaluated at {\textstyle z} . The convergence is uniform in {\textstyle z}  in the sense that

{\displaystyle \lim _{n\to \infty }\;\sup _{z\in \mathbb {R} }\;\left|\mathbb {P} \left[{\sqrt {n}}({\bar {X}}_{n}-\mu )\leq z\right]-\Phi \left({\frac {z}{\sigma }}\right)\right|=0~,}

where {\textstyle \sup }  denotes the least upper bound (or supremum)of the set

Let {\textstyle \{X_{1},\ldots ,X_{n}\}}  be a random sample of size {\textstyle n}  — that is, a sequence of independent and identically distributed (i.i.d.) random variables drawn from a distribution of expected value given by {\textstyle \mu }  and finite variance given by {\textstyle \sigma ^{2}} . Suppose we are interested in the sample average

{\displaystyle {\bar {X}}_{n}\equiv {\frac {X_{1}+\cdots +X_{n}}{n}}}

of these random variables. By the law of large numbers, the sample averages converge almost surely (and therefore also converge in probability) to the expected value {\textstyle \mu }  as {\textstyle n\to \infty } . The classical central limit theorem describes the size and the distributional form of the stochastic fluctuations around the deterministic number {\textstyle \mu }  during this convergence. More precisely, it states that as {\textstyle n}  gets larger, the distribution of the difference between the sample average {\textstyle {\bar {X}}_{n}}  and its limit {\textstyle \mu } , when multiplied by the factor {\textstyle {\sqrt {n}}}  (that is {\textstyle {\sqrt {n}}({\bar {X}}_{n}-\mu )} ) approximates the normal distribution with mean 0 and variance {\textstyle \sigma ^{2}} . For large enough n, the distribution of {\textstyle {\bar {X}}_{n}}  is close to the normal distribution with mean {\textstyle \mu }  and variance {\textstyle \sigma ^{2}/n} . The usefulness of the theorem is that the distribution of {\textstyle {\sqrt {n}}({\bar {X}}_{n}-\mu )}  approaches normality regardless of the shape of the distribution of the individual {\textstyle X_{i}} . Formally, the theorem can be stated as follows: