Statistics Papers

Document Type

Journal Article

Date of this Version

7-2014

Publication Source

The Annals of Applied Statistics

Volume

8

Issue

2

Start Page

1256

Last Page

1280

DOI

10.1214/14-AOAS739

Abstract

Most subjective probability aggregation procedures use a single probability judgment from each expert, even though it is common for experts studying real problems to update their probability estimates over time. This paper advances into unexplored areas of probability aggregation by considering a dynamic context in which experts can update their beliefs at random intervals. The updates occur very infrequently, resulting in a sparse data set that cannot be modeled by standard time-series procedures. In response to the lack of appropriate methodology, this paper presents a hierarchical model that takes into account the expert’s level of self-reported expertise and produces aggregate probabilities that are sharp and well calibrated both in- and out-of-sample. The model is demonstrated on a real-world data set that includes over 2300 experts making multiple probability forecasts over two years on different subsets of 166 international political events.

Copyright/Permission Statement

The original and published work is available at: https://projecteuclid.org/euclid.aoas/1404229533#abstract

Keywords

probability aggregation, dynamic linear model, hierarchical modeling, expert forecast, subjective probability, bias estimation, calibration, time series

Share

COinS
 

Date Posted: 27 November 2017

This document has been peer reviewed.