In this paper we study theoretical foundations for programming with bags. We fully determine the strength of many polynomial bag operators relative to an ambient query language. Then picking the strongest combination of these operators we obtain the yardstick nested bag query language NBL(monus, unique). The relationship between nested relational algebra and various fragments of NBL(monus, unique) is investigated. The precise amount of extra power that NBL(monus, unique) possesses over the nested relational algebra is determined. An ordering for dealing with partial information in bags is proposed and a technique for lifting a linear order at base types to linear order at all types is presented. This linear order is used to prove the conservative extension property for several bag languages. Using this property, we prove some inexpressibility results for NBL(monus, unique). In particular, it can not test for a property that is simultaneously infinite and co-infinite (for example, parity). Then non-polynomial primitives such as powerbag, structural recursion and bounded loop are studied. Structural recursion on bags is shown to be strictly more powerful than the powerbag primitive and it is equivalent to the bounded loop operator. Finally, we show that the numerical functions expressible in NBL(monus, unique) augmented by structural recursion are precisely the primitive recursive functions.