Perception-action loop is a fundamental concept in robotics that involves the iterative cycle of perceiving the environment and taking actions based on those perceptions. Various sensors and actuators are used in this process, and deep learning techniques have shown remarkable performance in improving the effectiveness of this cycle in various applications. However, previous works in this field have focused on solving research problems that are relevant to a single agent, and challenges for multi-robot systems with perception-action loops are much less addressed. This dissertation addresses the challenges in establishing perception-action loops for multi-robot systems. We examine three questions at increasing levels of abstraction. First, can the perception-action loop be closed with optimal action? Second, can multi-robot systems achieve this loop with dynamic objects? Third, can this loop be decentralized and scalable? The dissertation is divided into three main parts to answer these questions, and the perimeter defense game is used as an application. We investigate the use of deep learning techniques for coupling perception to action in multi-robot systems. Part I focuses on finding optimal action in a single-robot system. We aim to realize the system from theory to practice, and the performance discrepancy at large scales suggests the deeper study of multi-robot systems. Part II investigates achieving the perception-action loop in 2 vs. 1 multi-robot systems based on centralized policy. We employ vision sensors to perform state estimation and incorporate deep learning-based lightweight networks to enable multi-view perception. Part III introduces decentralized N vs. N multi-robot systems that are scalable based on local perception and communication. We design a learning framework based on graph neural networks in learning decentralized actions and demonstrate that our proposed networks can be trained at a small scale but generalized to large scales.