Stream Order and Order Statistics: Quantile Estimation in Random-Order Streams

Loading...
Thumbnail Image
Penn collection
Departmental Papers (CIS)
Degree type
Discipline
Subject
communication complexity
stochastically generated streams
stream computation
Funder
Grant number
License
Copyright date
Distributor
Related resources
Author
Mcgregor, Andrew
Contributor
Abstract

When trying to process a data stream in small space, how important is the order in which the data arrive? Are there problems that are unsolvable when the ordering is worst case, but that can be solved (with high probability) when the order is chosen uniformly at random? If we consider the stream as if ordered by an adversary, what happens if we restrict the power of the adversary? We study these questions in the context of quantile estimation, one of the most well studied problems in the data-stream model. Our results include an O(polylogn)-space, O(log log n)-pass algorithm for exact selection in a randomly ordered stream of n elements. This resolves an open question of Munro and Paterson [Theoret. Comput. Sci., 23 (1980), pp. 315-323]. We then demonstrate an exponential separation between the random-order and adversarial-order models: using O(polylog n) space, exact selection requires O(log n/log log n) passes in the adversarial-order model. This lower bound, in contrast to previous results, applies to fully general randomized algorithms and is established via a new bound on the communication complexity of a natural pointer-chasing style problem. We also prove the first fully general lower bounds in the random-order model:. finding an element with rank n/2 +/- n(delta) in the single-pass random-order model with probability at least 9/10 requires Omega(root n(1-3 delta)/log n) space.

Advisor
Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Publication date
2009-01-30
Journal title
Volume number
Issue number
Publisher
Publisher DOI
Journal Issue
Comments
Stream Order and Order Statistics: Quantile Estimation in Random-Order Streams Sudipto Guha and Andrew McGregor, SIAM J. Comput. 38, 2044 (2009), DOI:10.1137/07069328X Copyright SIAM, 2009. Reprinted in SIAM Journal on Computing, Volume 38, Issue 5, pages 2044-2059.
Recommended citation
Collection