Mutation is the ultimate source of the genetic variation—including genetic variation for mutation rate itself—that fuels evolution. Selection to increase the genomic mutation rate, driven by selective sweeps of beneficial mutations, can be strong and rapid where genetic linkage is present, as evidenced by numerous observations in experimental microbial populations. Selection to decrease the mutation rate, in contrast, is expected to depend on avoidance of mutational load and act over a longer time scale. In keeping with this latter expectation, there have been relatively few experimental observations of the evolution of reduced genomic mutation rates. Here, I report the rapid evolution of reduced mutation rates in hypermutable E. coli populations propagated at extremely small effective size—a circumstance under which selection is generally minimized. I hypothesize that high deleterious mutation pressure can strengthen indirect selection favoring lower mutation rates in these populations, and find both phenotypic and genotypic evidence to support this hypothesis. Additionally, I use simulations to analyze the effect of high deleterious mutation pressure on nascent neutral lineages that arise in an expanding asexual population and find that the spread of these lineages can be impaired. I discuss these results in the light of fates of novel mutations and point to future work that will involve studying the fates of adaptive mutation under high deleterious mutation pressure.