Learning-based model predictive control (MPC) is an increasingly prominent control paradigm in recent years. One primary approach in learning-based MPC is to leverage machine or deep learning tools to construct accurate dynamics models, which are then deployed into optimization-based control schemes. Although these frameworks can potentially provide significant improvements in terms of model accuracy and control performance, a number of key challenges need to be addressed before they can realize their full potential and prove reliable for real-world applications. In this dissertation, we present a novel learning-based control paradigm tailored for nonlinear systems whose dynamics are described by ordinary differential equations. The modular structure of the paradigm allows the seamless integration of its components into various modules within a planning and control framework, enhancing the performance and robustness of the overall system. The data-driven models within existing learning-based MPC frameworks aim to strike a balance between architectural complexity and expressiveness. However, many of these models either have elaborate architectures, or lack sufficient expressiveness to capture the intricate interactions and nonlinearities within the system dynamics. As the first contribution of this dissertation, we outline a novel learning-based MPC framework that addresses this trade-off. In particular, we construct an architecturally simple and accurate dynamics model by combining prior knowledge of the system with a data-driven component. This combined model is lightweight, making it highly compatible with optimization-based control frameworks such as nonlinear MPC. Through extensive simulations and physical experiments, we demonstrate that not only does the model achieves accurate state predictions, but the closed-loop system with our learning-based MPC framework also exhibits significant performance improvements over important baselines. Next, using an extension of the proposed framework, we tackle a realistic and challenging problem of a quadrotor team. When quadrotors fly in formation, their interactions often result in complex and undesirable aerodynamic disturbances, compromising their control performance. We demonstrate that with our learning-based MPC framework, the quadrotor team is able to counteract these complex aerodynamic disturbances, achieve unprecedented trajectory tracking performance, and fly in an exceptionally tight formation. In many learning-based MPC frameworks, the dynamics models are learned offline. This implies that the models often fail to account for the uncertainties encountered during deployment. Learning the models online is possible, but that typically requires substantial training data and computational resources. As the second contribution of this dissertation, we alleviate these shortcomings by developing an adaptive framework that boosts the sample efficiency and dynamic uncertainty compensation in learning-based MPC. By leveraging concepts from adaptive control in a novel way, we demonstrate through simulations and real-world experiments that our proposed framework effectively compensates for dynamic, time-varying uncertainties in a sample and computationally efficient manner. A third challenge of learning-based MPC frameworks, is that there are often little or no guarantees on the accuracy of the data-driven models. Deep learning tools such as Monte Carlo dropout and deep ensembles can provide uncertainty estimates, but they typically do not come with formal guarantees. While there are some methods that provide estimates with guarantees, these estimates may be too conservative, making them challenging to integrate into the controller design. As the third contribution of this dissertation, we leverage recent uncertainty quantification techniques to extract uncertainty estimates for the model predictions. These estimates are not overly conservative and they are equipped with probabilistic guarantees. Through simulations and case studies, we demonstrate that the incorporation of these uncertainty estimates in different aspects of the controller enhances the uncertainty awareness and robustness of the closed-loop system.