World Library  
Flag as Inappropriate
Email this Article

Method of moments (statistics)

Article Id: WHEBN0002175504
Reproduction Date:

Title: Method of moments (statistics)  
Author: World Heritage Encyclopedia
Language: English
Subject: Statistics, List of statistics articles, Estimating equations, Probabilistic design, L-moment
Collection: Fitting Probability Distributions
Publisher: World Heritage Encyclopedia

Method of moments (statistics)

In statistics, the method of moments is a method of estimation of population parameters. One starts with deriving equations that relate the population moments (i.e., the expected values of powers of the random variable under consideration) to the parameters of interest. Then a sample is drawn and the population moments are estimated from the sample. The equations are then solved for the parameters of interest, using the sample moments in place of the (unknown) population moments. This results in estimates of those parameters. The method of moments was introduced by Karl Pearson in 1894.


Suppose that the problem is to estimate k unknown parameters \theta_{1}, \theta_{2}, \dots, \theta_{k} characterizing the distribution f_{W}(w; \theta) of the random variable W. Suppose the first k moments of the true distribution (the "population moments") can be expressed as functions of the \thetas:

\mu_{1} \equiv E[W]=g_{1}(\theta_{1}, \theta_{2}, \dots, \theta_{k}) ,
\mu_{2} \equiv E[W^2]=g_{2}(\theta_{1}, \theta_{2}, \dots, \theta_{k}) ,
\mu_{k} \equiv E[W^k]=g_{k}(\theta_{1}, \theta_{2}, \dots, \theta_{k}) .

Suppose a sample of size n is drawn, resulting in the values w_1, \dots, w_n. For j=1,\dots,k, let

\hat{\mu}_{j}=\frac{1}{n}\sum_{i=1}^{n} w_{i}^{j}

be the j-th sample moment, an estimate of \mu_{j}. The method of moments estimator for \theta_{1}, \theta_{2}, \dots, \theta_{k} denoted by \hat{\theta}_{1}, \hat{\theta}_{2}, \dots, \hat{\theta}_{k} is defined as the solution (if there is one) to the equations:

\hat \mu_{1} = g_{1}(\hat{\theta}_{1}, \hat{\theta}_{2}, \dots, \hat{\theta}_{k}) ,
\hat \mu_{2} = g_{2}(\hat{\theta}_{1}, \hat{\theta}_{2}, \dots, \hat{\theta}_{k}) ,
\hat \mu_{k} = g_{k}(\hat{\theta}_{1}, \hat{\theta}_{2}, \dots, \hat{\theta}_{k}) .

Advantages and disadvantages of this method

The method of moments is fairly simple and yields consistent estimators (under very weak assumptions), though these estimators are often biased.

In some respects, when estimating parameters of a known family of probability distributions, this method was superseded by Fisher's method of maximum likelihood, because maximum likelihood estimators have higher probability of being close to the quantities to be estimated and are more often unbiased.

However, in some cases, as in the above example of the gamma distribution, the likelihood equations may be intractable without computers, whereas the method-of-moments estimators can be quickly and easily calculated by hand.

Estimates by the method of moments may be used as the first approximation to the solutions of the likelihood equations, and successive improved approximations may then be found by the Newton–Raphson method. In this way the method of moments and the method of maximum likelihood are symbiotic.

In some cases, infrequent with large samples but not so infrequent with small samples, the estimates given by the method of moments are outside of the parameter space; it does not make sense to rely on them then. That problem never arises in the method of maximum likelihood. Also, estimates by the method of moments are not necessarily sufficient statistics, i.e., they sometimes fail to take into account all relevant information in the sample.

When estimating other structural parameters (e.g., parameters of a utility function, instead of parameters of a known probability distribution), appropriate probability distributions may not be known, and moment-based estimates may be preferred to maximum likelihood estimation.

See also

This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.

Copyright © World Library Foundation. All rights reserved. eBooks from World Library are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.