Topic Modeling: Optimal Estimation, Statistical Inference, And Beyond

Loading...
Thumbnail Image
Degree type
Doctor of Philosophy (PhD)
Graduate group
Statistics
Discipline
Subject
High-dimensional statistics
Machine learning
Optimality
Topic modeling
Statistics and Probability
Funder
Grant number
License
Copyright date
2022-10-05T20:22:00-07:00
Distributor
Related resources
Author
Wu, Ruijia
Contributor
Abstract

With the development of computer technology and the internet, increasingly large amounts of textual data are generated and collected every day. It is a significant challenge to analyze and extract meaningful and actionable information from vast amounts of unstructured textual data. This thesis explores several problems in topic modelings and provides new algorithms with theoretical guarantees. The first part of this thesis aims to develop an optimality theory for unsupervised topic modeling under the probabilistic latent semantic indexing (pLSI) model. Novel and computationally fast algorithms for estimation and inference of both the word-topic matrix and the topic-document matrix are proposed and their theoretical properties are investigated. Moreover, a refitting algorithm is proposed to establish asymptotic normality and construct valid confidence intervals for the individual entries of the word-topic and topic-document matrices. In the second part, we study supervised topic modeling, which jointly considers a collection of documents and their paired side information. To take account of the compositional nature of the topic-document matrix, we adapt the log-contrast model and introduce a novel bias-adjusted algorithm to investigate the regression coefficients in the generalized linear model. In addition, a de-biased procedure is proposed to establish an asymptotically unbiased and normally distributed estimator, and hence valid confidence intervals are constructed for the individual entries of regression coefficients. We also investigate the errors-in-variables models under the generalized linear model framework in the third part. We proposed an estimator when the measurement error is small.

Advisor
Tony Cai
Date of degree
2022-01-01
Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Volume number
Issue number
Publisher
Publisher DOI
Journal Issue
Comments
Recommended citation