We show that long-range ferroelectric and incommensurate magnetic order appear simultaneously in a single phase transition in Ni3V2O8. The temperature and magnetic-field dependence of the spontaneous polarization show a strong coupling between magnetic and ferroelectric orders. We determine the magnetic symmetry using Landau theory for continuous phase transitions, which shows that the spin structure alone can break spatial inversion symmetry leading to ferroelectric order. This phenomenological theory explains our experimental observation that the spontaneous polarization is restricted to lie along the crystal b axis and predicts that the magnitude should be proportional to a magnetic order parameter.