World Library  
Flag as Inappropriate
Email this Article

Empirical distribution function

Article Id: WHEBN0002690455
Reproduction Date:

Title: Empirical distribution function  
Author: World Heritage Encyclopedia
Language: English
Subject: Kolmogorov–Smirnov test, Dvoretzky–Kiefer–Wolfowitz inequality, Confidence and prediction bands, Kaplan–Meier estimator, Statistics
Collection: Data Analysis, Empirical Process, Nonparametric Statistics
Publisher: World Heritage Encyclopedia

Empirical distribution function

The blue line shows an empirical distribution function. The black bars represent the samples corresponding to the empirical distribution function and the gray curve is the true cumulative distribution function.

In statistics, the empirical distribution function is the distribution function associated with the empirical measure of the sample. This cumulative distribution function is a step function that jumps up by 1/n at each of the n data points. The empirical distribution function estimates the cumulative distribution function underlying of the points in the sample and converges with probability 1 according to the Glivenko–Cantelli theorem. A number of results exist to quantify the rate of convergence of the empirical distribution function to the underlying cumulative distribution function.


  • Definition 1
  • Asymptotic properties 2
  • See also 3
  • References 4
  • Further reading 5
  • External links 6


Let (x1, …, xn) be independent, identically distributed real random variables with the common cumulative distribution function F(t). Then the empirical distribution function is defined as [1][2]

\hat F_n(t) = \frac{ \mbox{number of elements in the sample} \leq t}n = \frac{1}{n} \sum_{i=1}^n \mathbf{1}_{x_i \le t},

where \mathbf{1}_{A} is the indicator of event A. For a fixed t, the indicator \mathbf{1}_{x_i \le t} is a Bernoulli random variable with parameter p = F(t), hence \scriptstyle n \hat F_n(t) is a binomial random variable with mean nF(t) and variance . This implies that \scriptstyle \hat F_n(t) is an unbiased estimator for F(t).

However, in some textbooks,[3][4] the definition is given as \hat F_n(t) = \frac{1}{n+1} \sum_{i=1}^n \mathbf{1}_{x_i \le t}

Asymptotic properties

Since the ratio (n+1) / n approaches 1 as n goes to infinity, the asymptotic properties of the two definitions that are given above are the same.

By the strong law of large numbers, the estimator \scriptstyle\hat{F}_n(t) converges to F(t) as almost surely, for every value of t:[1]

\hat F_n(t)\ \xrightarrow{a.s.}\ F(t),

thus the estimator \scriptstyle\hat{F}_n(t) is consistent. This expression asserts the pointwise convergence of the empirical distribution function to the true cumulative distribution function. There is a stronger result, called the Glivenko–Cantelli theorem, which states that the convergence in fact happens uniformly over t:[5]

\|\hat F_n-F\|_\infty \equiv \sup_{t\in\mathbb{R}} \big|\hat F_n(t)-F(t)\big|\ \xrightarrow{a.s.}\ 0.

The sup-norm in this expression is called the Kolmogorov–Smirnov statistic for testing the goodness-of-fit between the empirical distribution \scriptstyle\hat{F}_n(t) and the assumed true cumulative distribution function F. Other norm functions may be reasonably used here instead of the sup-norm. For example, the L²-norm gives rise to the Cramér–von Mises statistic.

The asymptotic distribution can be further characterized in several different ways. First, the central limit theorem states that pointwise, \scriptstyle\hat{F}_n(t) has asymptotically normal distribution with the standard \sqrt{n} rate of convergence:[1]

\sqrt{n}\big(\hat F_n(t) - F(t)\big)\ \ \xrightarrow{d}\ \ \mathcal{N}\Big( 0, F(t)\big(1-F(t)\big) \Big).

This result is extended by the Donsker’s theorem, which asserts that the empirical process \scriptstyle\sqrt{n}(\hat{F}_n - F), viewed as a function indexed by \scriptstyle t\in\mathbb{R}, converges in distribution in the Skorokhod space \scriptstyle D[-\infty, +\infty] to the mean-zero Gaussian process \scriptstyle G_F = B \circ F, where B is the standard Brownian bridge.[5] The covariance structure of this Gaussian process is

\mathrm{E}[\,G_F(t_1)G_F(t_2)\,] = F(t_1\wedge t_2) - F(t_1)F(t_2).

The uniform rate of convergence in Donsker’s theorem can be quantified by the result known as the Hungarian embedding:[6]

\limsup_{n\to\infty} \frac{\sqrt{n}}{\ln^2 n} \big\| \sqrt{n}(\hat F_n-F) - G_{F,n}\big\|_\infty < \infty, \quad \text{a.s.}

Alternatively, the rate of convergence of \scriptstyle\sqrt{n}(\hat{F}_n-F) can also be quantified in terms of the asymptotic behavior of the sup-norm of this expression. Number of results exist in this venue, for example the Dvoretzky–Kiefer–Wolfowitz inequality provides bound on the tail probabilities of \scriptstyle\sqrt{n}\|\hat{F}_n-F\|_\infty:[6]

\Pr\!\Big( \sqrt{n}\|\hat{F}_n-F\|_\infty > z \Big) \leq 2e^{-2z^2}.

In fact, Kolmogorov has shown that if the cumulative distribution function F is continuous, then the expression \scriptstyle\sqrt{n}\|\hat{F}_n-F\|_\infty converges in distribution to \scriptstyle\|B\|_\infty, which has the Kolmogorov distribution that does not depend on the form of F.

Another result, which follows from the law of the iterated logarithm, is that [6]

\limsup_{n\to\infty} \frac{\sqrt{n}\|\hat{F}_n-F\|_\infty}{\sqrt{2\ln\ln n}} \leq \frac12, \quad \text{a.s.}


\liminf_{n\to\infty} \sqrt{2n\ln\ln n} \|\hat{F}_n-F\|_\infty = \frac{\pi}{2}, \quad \text{a.s.}

See also


  1. ^ a b c van der Vaart, A.W. (1998). Asymptotic statistics. Cambridge University Press. p. 265.  
  2. ^ PlanetMath
  3. ^ Coles, S. (2001) An Introduction to Statistical Modeling of Extreme Values. Springer, p. 36, Definition 2.4. ISBN 978-1-4471-3675-0.
  4. ^ Madsen, H.O., Krenk, S., Lind, S.C. (2006) Methods of Structural Safety. Dover Publications. p. 148-149. ISBN 0486445976
  5. ^ a b van der Vaart, A.W. (1998). Asymptotic statistics. Cambridge University Press. p. 266.  
  6. ^ a b c van der Vaart, A.W. (1998). Asymptotic statistics. Cambridge University Press. p. 268.  

Further reading

  • Shorack, G.R.; Wellner, J.A. (1986). Empirical Processes with Applications to Statistics. New York: Wiley.  

External links

This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.

Copyright © World Library Foundation. All rights reserved. eBooks from World Library are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.