We examine the elastic and vibrational properties of spring lattices, including the two-dimensional square and kagome and the three-dimensional cubic and pyrochlore lattices, which are at the verge of mechanical instability due to under-coordination. By using the extended Maxwell counting argument, we are able to count the number of soft phonon modes in these systems. These modes are stabilized by adding additional springs, bending energy terms or isotropic tension. By tuning the strength of these terms, we are able to continuously approach the mechanical instability to obtain scaling laws for the elastic moduli, as well as critical length and frequency scales. Further, these lattices can be deformed along their soft modes to obtain entire families of lattices with novel soft elastic properties. We find models of zero bulk modulus elasticity, leading to a negative Poisson ratio and a conformally invariant elasticity theory, whose implications we discuss. By following the mechanical instability, we obtain a full phase diagram of the isostatic point in these systems and encounter an unusual phase. In this phase, the ground state has a subextensive entropy, but at low temperatures the entropy of phonon fluctuations selects an ordered thermodynamic phase, through an elastic order-by-disorder effect. These properties are also compared to an elastic Ising antiferromagnet on a triangular lattice. We relate our models to physical systems, such as the jamming of granular media, semi-flexible polymer gels and a quasi-two-dimensional colloidal experiment, and discuss their implications.