A New Approach To Multiple Hypothesis Testing: Finite-Sample Strong Control With Arbitrary Dependency

Loading...
Thumbnail Image
Degree type
Doctor of Philosophy (PhD)
Graduate group
Applied Mathematics
Discipline
Subject
arbitrary dependency
large sample
Multiple Hypothesis Testing
small sample
Applied Mathematics
Finance and Financial Management
Statistics and Probability
Funder
Grant number
License
Copyright date
2021-08-31T20:20:00-07:00
Distributor
Related resources
Author
Zhang, Gengyuan
Contributor
Abstract

Multiple (simultaneous) hypothesis testing is key for avoiding "data dredging'' with repeated single hypothesis testing across competing hypotheses. But, existing multiple testing procedures (MTP's) face serious limitations. In small (finite) samples common to social sciences, existing MTP's assume independent statistics, e.g., uncorrelated returns to different trading strategies. They produce conservative control of Type I errors in weak control problems (where all nulls are true) and limited power in strong control problems (with a mixture of true and false nulls). Large (asymptotic) sample procedures allow for general dependence, greatly expanding testing applications. But they fail to produce control in smaller samples, or when the number of tests increases with the amount of data. They face other technical limitations and are otherwise unrelated to small sample procedures. We present a new method (NM) MTP and a generalized sequential step-down version (NM-GS) that produces strong control and accurate weak control even in a small sample with an assumed but arbitrary dependency of test statistics. NM and NM-GS nest previous small-sample MTP's. Assumed dependency can be replaced with an empirical sample estimate in a large sample, producing control similar to existing procedures but with fewer assumptions. Being parametric, obtaining control requires less data relative to the number of hypotheses. NM and NM-GS create a unified framework between small and large samples, even supporting hybrid solutions.

Advisor
Kent Smetters
Date of degree
2020-01-01
Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Volume number
Issue number
Publisher
Publisher DOI
Journal Issue
Comments
Recommended citation