Statistics Papers

Document Type

Technical Report

Date of this Version


Publication Source






Start Page


Last Page





An effect modifier is a pretreatment covariate that affects the magnitude of the treatment effect or its stability. When there is effect modification, an overall test that ignores an effect modifier may be more sensitive to unmeasured bias than a test that combines results from subgroups defined by the effect modifier. If there is effect modification, one would like to identify specific subgroups for which there is evidence of effect that is insensitive to small or moderate biases. In this paper, we propose an exploratory method for discovering effect modification, and combine it with a confirmatory method of simultaneous inference that strongly controls the familywise error rate in a sensitivity analysis, despite the fact that the groups being compared are defined empirically. A new form of matching, strength-k matching, permits a search through more than k covariates for effect modifiers, in such a way that no pairs are lost, provided that at most k covariates are selected to group the pairs. In a strength-k match, each set of k covariates is exactly balanced, although a set of more than k covariates may exhibit imbalance. We apply the proposed method to study the effects of the earthquake that struck Chile in 2010.

Copyright/Permission Statement

This is a pre-copyedited, author-produced PDF of an article accepted for publication in Biometrika following peer review. The version of record [Hsu, J.Y., Zubizarreta, J.R., Small, D.S., & Rosenbaum, P.R. Strong Control of the Familywise Error Rate in Observational Studies that Discover Effect Modification by Exploratory Methods. Biometrika 102, no. 4: pp. 767-782] is available online at:


design sensitivity, effect modification, integer programming, matched sampling, power of a sensitivity analysis, observational study, sensitivity analysis, truncated product of P-values



Date Posted: 25 October 2018

This document has been peer reviewed.