Large-scale, distributed cyber-physical systems are common in everyday life; just a few of the critical applications include vehicles, automated factories and warehouses, the electric grid, electricity generation plants, satellites, water purification plants, and sewage management systems. Unfortunately, systems deployed in all of these critical applications and others are subject to faults; that is, components in the system can experience failure due to software bugs, broken actuators, miscalibrated sensors, or even adversarial attacks. Damages caused by many faults in CPS have immediate, real-world consequences. For instance, in 2021, at an automated grocery packing facility in London, crashed robots caused a fire that endangered lives, harmed the environment, and ultimately resulted in £35 million in damages. This thesis considers how to design new fault detection and recovery techniques for large-scale, distributed cyber-physical systems. Across a variety of different CPS, developed techniques will demonstrate that resource-efficient, and in many cases bounded-time, methods of fault detection and recovery can be developed by leveraging the ‘physical’ part of cyber-physical systems. The discussion will start by considering rotor faults in aerial systems with dozens of rotors, then move to compute node faults in embedded control systems like that of a passenger vehicle, and finally consider compute node faults in multi-robot systems engaged in a collective action like flocking. In each case, insights gleaned from the physical nature of thesystem will be used to develop a fault detection and recovery approach. The result is a set of new fault detection and recovery techniques, and corresponding insights, that system designers can use to make systems that are safe and secure despite large scale, general fault models, and limited resources.