Revisiting the Direct Sum Theorem and Space Lower Bounds in Random Order Streams

Loading...
Thumbnail Image
Penn collection
Departmental Papers (CIS)
Degree type
Discipline
Subject
data streams
algorithms
lower bounds
communication complexity
Funder
Grant number
License
Copyright date
Distributor
Related resources
Contributor
Abstract

Estimating frequency moments and $L_p$ distances are well studied problems in the adversarial data stream model and tight space bounds are known for these two problems. There has been growing interest in revisiting these problems in the framework of random-order streams. The best space lower bound known for computing the $k^{th}$ frequency moment in random-order streams is $\Omega(n^{1-2.5/k})$ by Andoni et al., and it is conjectured that the real lower bound shall be $\Omega(n^{1-2/k})$. In this paper, we resolve this conjecture. In our approach, we revisit the direct sum theorem developed by Bar-Yossef et al. in a random-partition private messages model and provide a tight $\Omega(n^{1-2/k}/\ell)$ space lower bound for any $\ell$-pass algorithm that approximates the frequency moment in random-order stream model to a constant factor. Finally, we also introduce the notion of space-entropy tradeoffs in random order streams, as a means of studying intermediate models between adversarial and fully random order streams. We show an almost tight space-entropy tradeoff for $L_\infty$ distance and a non-trivial tradeoff for $L_p$ distances.

Advisor
Date of presentation
2009-04-27
Conference name
Departmental Papers (CIS)
Conference dates
2023-05-17T02:52:35.000
Conference location
Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Volume number
Issue number
Publisher
Publisher DOI
Journal Issue
Comments
Recommended citation
Collection