The Perils of Randomization Checks in the Analysis of Experiments

Loading...
Thumbnail Image
Penn collection
Departmental Papers (ASC)
Degree type
Discipline
Subject
balance test
control
covariate
estimator
regression
variance
Communication
Social and Behavioral Sciences
Funder
Grant number
License
Copyright date
Distributor
Related resources
Contributor
Abstract

In the analysis of experimental data, randomization checks, also known as balance tests, are used to indicate whether a randomization has produced balance on various characteristics across experimental conditions. Randomization checks are popular in many fields although their merits have yet to be established. The grounds on which balance tests are generally justified include either 1) the credibility of experimental findings, and/or 2) the efficiency of the statistical model. We show that balance tests cannot improve either credibility or efficiency. The most common “remedy” resulting from a failed balance test is the inclusion as a covariate of a variable failing the test; this practice cannot improve the choice of statistical model. Other commonly suggested responses to failed balance tests such as post-stratification or re-randomization also fail to improve on methods that do not require balance tests. We advocate resisting reviewer requests for randomization checks in all but some narrowly defined circumstances

Advisor
Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Publication date
2012-01-01
Volume number
Issue number
Publisher
Publisher DOI
Journal Issue
Comments
Recommended citation
Collection