We investigate a stable Casimir force configuration consisting of an object contained inside a spherical or spheroidal cavity filled with a dielectric medium. The spring constant for displacements from the center of the cavity and the dependence of the energy on the relative orientations of the inner object and the cavity walls are computed. We find that the stability of the force equilibrium—unlike the direction of the torque—can be predicted based on the sign of the force between two slabs of the same material.