World Library  
Flag as Inappropriate
Email this Article

Randomized experiment

Article Id: WHEBN0006033300
Reproduction Date:

Title: Randomized experiment  
Author: World Heritage Encyclopedia
Language: English
Subject: Experiment, List of important publications in statistics, Generalized randomized block design, Restricted randomization, Oregon Medicaid health experiment
Collection: Design of Experiments, Experiments
Publisher: World Heritage Encyclopedia

Randomized experiment

Flowchart of four phases (enrollment, intervention allocation, follow-up, and data analysis) of a parallel randomized trial of two groups, modified from the CONSORT 2010 Statement[1]

In science, randomized experiments are the experiments that allow the greatest reliability and validity of statistical estimates of treatment effects. Randomization-based inference is especially important in experimental design and in survey sampling.


  • Overview 1
  • Online Randomized Controlled Experiments 2
  • History 3
  • Statistical Interpretation 4
  • Empirical evidence that randomization makes a difference 5
  • See also 6
  • References 7


In the statistical theory of design of experiments, randomization involves randomly allocating the experimental units across the treatment groups. For example, if an experiment compares a new drug against a standard drug, then the patients should be allocated to either the new drug or to the standard drug control using randomization.

Randomized experimentation is not haphazard. Randomization reduces bias by equalising other factors that have not been explicitly accounted for in the experimental design (according to the law of large numbers). Randomization also produces ignorable designs, which are valuable in model-based statistical inference, especially Bayesian or likelihood-based. In the design of experiments, the simplest design for comparing treatments is the "completely randomized design". Some "restriction on randomization" can occur with blocking and experiments that have hard-to-change factors; additional restrictions on randomization can occur when a full randomization is infeasible or when it is desirable to reduce the variance of estimators of selected effects.

Randomization of treatment in clinical trials pose ethical problems. In some cases, randomization reduces the therapeutic options for both physician and patient, and so randomization requires clinical equipoise regarding the treatments.

Online Randomized Controlled Experiments

Web sites can run randomized controlled experiments [2] to create a feedback loop.[3] Key differences between offline experimentation and online experiments include:[3][4]

  • Logging: user interactions can be logged reliably.
  • Number of users: large sites, such as Amazon, Bing/Microsoft, and Google run experiments, each with over a million users.
  • Number of concurrent experiments: large sites run tens of overlapping, or concurrent, experiments.[5]
  • Robots, whether web crawlers from valid sources or malicious internet bots.
  • Ability to ramp-up experiments from low percentages to higher percentages.
  • Speed / performance has significant impact on key metrics.[3][6]
  • Ability to use the pre-experiment period as an A/A test to reduce variance.[7]


The earliest controlled experiment appears to have been suggested in the Old Testament's Book of Daniel. King Nebuchadnezzar proposed that some Israelites eat "a daily amount of food and wine from the king's table." Daniel preferred a vegetarian diet, but the official was concerned that the king would "see you looking worse than the other young men your age? The king would then have my head because of you." Daniel then proposed the following controlled experiment: "Test your servants for ten days. Give us nothing but vegetables to eat and water to drink. Then compare our appearance with that of the young men who eat the royal food, and treat your servants in accordance with what you see” (Daniel 1, 12– 13).[8][9]

Randomized experiments were institutionalized in psychology and education in the late eighteen-hundreds, following the invention of randomized experiments by C. S. Peirce.[10][11][12][13] Outside of psychology and education, randomized experiments were popularized by R.A. Fisher in his book Statistical Methods for Research Workers, which also introduced additional principles of experimental design.

Statistical Interpretation

The Rubin Causal Model provides a common way to describe a randomized experiment. While the Rubin Causal Model provides a framework for defining the causal parameters (i.e., the effects of a randomized treatment on an outcome), the analysis of experiments can take a number of forms. Most commonly, randomized experiments are analyzed using ANOVA, Student's t-test, Regression analysis, or a similar statistical test.

Empirical evidence that randomization makes a difference

Empirically differences between randomized and non-randomized studies,[14] and between adequately and inadequately randomized trials have been difficult to detect [15] .[16]

See also


  1. ^
  2. ^
  3. ^ a b c
  4. ^
  5. ^
  6. ^
  7. ^
  8. ^
  9. ^
  10. ^
  11. ^
  12. ^
  13. ^
  14. ^
  15. ^
  16. ^

This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.

Copyright © World Library Foundation. All rights reserved. eBooks from World Library are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.