It is widely known that conditional covariances of asset returns change over time. Researchers adopt many strategies to accommodate conditional heteroskedasticity. Among the most popular: (a) chopping the data into short blacks of time and assuming homoskedasticity within the blocks, (b) performing one-sided rolling regressions, in which only data from, say, the preceding five year period is used to estimate the conditional covariance of returns at a given date, and (c) two-sided rolling regressions which use, say, five years of leads and five years of lags. GARCH amounts to a one-sided rolling regression with exponentially declining weights. We derive asymptotically optimal window lengths for standard rolling regressions and optimal weights for weighted rolling regressions. An empirical model of the S&P 500 stock index provides and example.