In this paper we present a new scheme for the representation of object surfaces. The purpose is to model a surface efficiently in a coarse to fine hierarchy. Our scheme is based on the combination of spherical harmonic functions and wavelet networks on the sphere. The coefficients can be estimated from scattered data sampled from a star-shaped object’s surface. Spherical harmonic functions are used to model the coarse structure of the surface, while spherical Gabor wavelets are used for the representation of fine scale detail. Theoretical background on wavelets on the sphere is provided as well as a discussion of implementation issues concerning convolutions on the sphere. Results are presented which show the efficiency of the proposed representation.