Date of Award

2020

Degree Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Graduate Group

Applied Mathematics

First Advisor

Kent Smetters

Abstract

Multiple (simultaneous) hypothesis testing is key for avoiding "data dredging'' with repeated single hypothesis testing across competing hypotheses. But, existing multiple testing procedures (MTP's) face serious limitations. In small (finite) samples common to social sciences, existing MTP's assume independent statistics, e.g., uncorrelated returns to different trading strategies. They produce conservative control of Type I errors in weak control problems (where all nulls are true) and limited power in strong control problems (with a mixture of true and false nulls). Large (asymptotic) sample procedures allow for general dependence, greatly expanding testing applications. But they fail to produce control in smaller samples, or when the number of tests increases with the amount of data. They face other technical limitations and are otherwise unrelated to small sample procedures.

We present a new method (NM) MTP and a generalized sequential step-down version (NM-GS) that produces strong control and accurate weak control even in a small sample with an assumed but arbitrary dependency of test statistics. NM and NM-GS nest previous small-sample MTP's. Assumed dependency can be replaced with an empirical sample estimate in a large sample, producing control similar to existing procedures but with fewer assumptions. Being parametric, obtaining control requires less data relative to the number of hypotheses. NM and NM-GS create a unified framework between small and large samples, even supporting hybrid solutions.

Download

Embargoed

Available to all on Saturday, June 03, 2023

Share

COinS