The future of the Internet of Things (IoT) envisions very large networks of spatially distributed devices cooperating to solve common tasks in both industrial and urban environments. That vision fundamentally relies on the use of wireless communication across such networks to enable flexibility, mobility, and dynamic configurations. The use of wireless connectivity in place of traditional wired communication to handle communication between those components, however, adds particular challenges to the design of control, optimization and communication policies. Control systems, in particular, are traditionally designed under the assumption that communication between spatially distributed components is fast and reliable --- which might not necessarily hold if the different components of the control system communicate over a wireless network. That is because wireless networks are characterized by rapidly changing transmission conditions and are further subject to packet losses as the transmission conditions in the channel deteriorate. That, in turn, implies that components of a wireless control system might have to operate under missing or outdated information. Communication policies for standard wireless networks, on the other hand, typically aim to optimize traditional communication metrics that might not necessarily optimize the performance of the specific application taking place over that network. Hence, in the first part of this work we present a constrained reinforcement learning framework for the joint design of control and communication policies that optimizes the performance of the control system while satisfying requirements on the usage of communication resources. As the scale of deployment of spatially distributed control systems grows, however, it becomes of paramount importance to design communication policies that not only minimize interference among concurrent transmissions and optimize the performance of the underlying application taking place over that network, but that are also scalable. In the second part of this dissertation we thus investigate the design of scalable resource allocation policies for wireless control systems. By incorporating the structure of the communication network into the parameterization used to represent the communication policy, we show that the proposed approach yields resource allocation policies that not only outperform hand-crafted solutions, but that can also be successfully transferred across networks of varying size. While the first two parts of this dissertation focus on the design of control and communication policies in wireless control systems, we study more general decision processes over networks in the third part of this work. In particular, we propose a meta-learning approach to solve distributed optimization problems and show that one can learn distributed optimization algorithms that can match or exceed the performance of hand-crafted ones.