Technical Reports (CIS)

Document Type

Technical Report

Date of this Version

January 2004

Comments

University of Pennsylvania Department of Computer and Information Science Technical Report No. MS-CIS-04-07.

Abstract

Over the last decade there has been growing interest in pool-based active learning techniques, where instead of receiving an i.i.d. sample from a pool of unlabeled data, a learner may take an active role in selecting examples from the pool. Queries to an oracle (a human annotator in most applications) provide label information for the selected observations, but at a cost. The challenge is to end up with a model that provides the best possible generalization error at the least cost. Popular methods such as uncertainty sampling often work well, but sometimes fail badly. We take the A-optimality criterion used in optimal experimental design, and extend it so that it can be used for pool-based active learning of logistic regression classifiers. A-optimality has attractive theoretical properties, and empirical evaluation confirms that it offers a more robust approach to active learning for logistic regression than alternatives.

Keywords

computer science, active learning, logistic regression

Share

COinS
 

Date Posted: 19 August 2004