We argue in this paper that many common adverbial phrases generally taken to be discourse connectives signalling discourse relations between adjacent discourse units are instead anaphors. We do this by (i) demonstrating their behavioral similarity with more common anaphors (pronouns and definite NPs); (ii) presenting a general framework for understanding anaphora into which they nicely fit; (iii) showing the interpretational benefits of understanding discourse adverbials as anaphors; and (iv) sketching out a lexicalised grammar that facilitates discourse interpretation as a product of compositional rules, anaphor resolution and inference.