Strong Control of the Familywise Error Rate in Observational Studies that Discover Effect Modification by Exploratory Methods

Thumbnail Image
Penn collection
Statistics Papers
Degree type
design sensitivity
effect modification
integer programming
matched sampling
power of a sensitivity analysis
observational study
sensitivity analysis
truncated product of P-values
Business Analytics
Design of Experiments and Sample Surveys
Management Sciences and Quantitative Methods
Statistical Models
Statistics and Probability
Grant number
Copyright date
Related resources
Hsu, Jesse Y
Zubizarreta, José R
Small, Dylan S
Rosenbaum, Paul R

An effect modifier is a pretreatment covariate that affects the magnitude of the treatment effect or its stability. When there is effect modification, an overall test that ignores an effect modifier may be more sensitive to unmeasured bias than a test that combines results from subgroups defined by the effect modifier. If there is effect modification, one would like to identify specific subgroups for which there is evidence of effect that is insensitive to small or moderate biases. In this paper, we propose an exploratory method for discovering effect modification, and combine it with a confirmatory method of simultaneous inference that strongly controls the familywise error rate in a sensitivity analysis, despite the fact that the groups being compared are defined empirically. A new form of matching, strength-k matching, permits a search through more than k covariates for effect modifiers, in such a way that no pairs are lost, provided that at most k covariates are selected to group the pairs. In a strength-k match, each set of k covariates is exactly balanced, although a set of more than k covariates may exhibit imbalance. We apply the proposed method to study the effects of the earthquake that struck Chile in 2010.

Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Publication date
Volume number
Issue number
Publisher DOI
Journal Issue
Recommended citation