We develop a unified framework to classify topological defects in insulators and superconductors described by spatially modulated Bloch and Bogoliubov de Gennes Hamiltonians.We consider Hamiltonians H(k,r) that vary slowly with adiabatic parameters r surrounding the defect and belong to any of the ten symmetry classes defined by time-reversal symmetry and particle-hole symmetry. The topological classes for such defects are identified and explicit formulas for the topological invariants are presented. We introduce a generalization of the bulk-boundary correspondence that relates the topological classes to defect Hamiltonians to the presence of protected gapless modes at the defect. Many examples of line and point defects in three-dimensional systems will be discussed. These can host one dimensional chiral Dirac fermions, helical Dirac fermions, chiral Majorana fermions, and helical Majorana fermions, as well as zero-dimensional chiral and Majorana zero modes. This approach can also be used to classify temporal pumping cycles, such as the Thouless charge pump, as well as a fermion parity pump, which is related to the Ising non-Abelian statistics of defects that support Majorana zero modes.