We evaluate the high temperature limit of the free energy of spin glasses on the hypercube with Hamiltonian HN(σ) = σTJσ, where the coupling matrix J is drawn from certain symmetric orthogonally invariant ensembles. Our derivation relates the annealed free energy of these models to a spherical integral, and expresses the limit of the free energy in terms of the limiting spectral measure of the coupling matrix J. As an application, we derive the limiting free energy of the random orthogonal model at high temperatures, which confirms non-rigorous calculations of Marinari et al. (J Phys A 27:7647, 1994). Our methods also apply to other well-known models of disordered systems, including the SK and Gaussian Hopfield models.