Date of Award
2020
Degree Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Graduate Group
Applied Mathematics
First Advisor
Kent Smetters
Abstract
Multiple (simultaneous) hypothesis testing is key for avoiding "data dredging'' with repeated single hypothesis testing across competing hypotheses. But, existing multiple testing procedures (MTP's) face serious limitations. In small (finite) samples common to social sciences, existing MTP's assume independent statistics, e.g., uncorrelated returns to different trading strategies. They produce conservative control of Type I errors in weak control problems (where all nulls are true) and limited power in strong control problems (with a mixture of true and false nulls). Large (asymptotic) sample procedures allow for general dependence, greatly expanding testing applications. But they fail to produce control in smaller samples, or when the number of tests increases with the amount of data. They face other technical limitations and are otherwise unrelated to small sample procedures.
We present a new method (NM) MTP and a generalized sequential step-down version (NM-GS) that produces strong control and accurate weak control even in a small sample with an assumed but arbitrary dependency of test statistics. NM and NM-GS nest previous small-sample MTP's. Assumed dependency can be replaced with an empirical sample estimate in a large sample, producing control similar to existing procedures but with fewer assumptions. Being parametric, obtaining control requires less data relative to the number of hypotheses. NM and NM-GS create a unified framework between small and large samples, even supporting hybrid solutions.
Recommended Citation
Zhang, Gengyuan, "A New Approach To Multiple Hypothesis Testing: Finite-Sample Strong Control With Arbitrary Dependency" (2020). Publicly Accessible Penn Dissertations. 3958.
https://repository.upenn.edu/edissertations/3958
Embargoed
Available to all on Saturday, June 03, 2023Included in
Applied Mathematics Commons, Finance and Financial Management Commons, Statistics and Probability Commons