We perform a symmetry analysis of the zero-temperature instabilities of the tetragonal phase of Ca3Mn2O7 and Ca3Ti2O7 which is stable at high temperature. We introduce order parameters to characterize each of the possible lattice distortions to construct a Landau free energy which elucidates the proposed group-subgroup relations for structural transitions in these systems. We include the coupling between the unstable distortion modes and the macroscopic strain tensor. We also analyze the symmetry of the dominantly antiferromagnetic ordering which allows weak ferromagnetism. We show that in this phase the weak ferromagnetic moment and the spontaneous ferroelectric polarization are coupled, so that by rotating one of these orderings by applying an external electric or magnetic field one can rotate the other ordering. We discuss the number of different domains (including phase domains) which exist in each of the phases and indicate how these may be observed. First-principles calculations of Yildirim corroborate our assertion that domain walls in the nonferroelectric phase are narrow.