This thesis investigates algorithms for environments where nuanced, multidimensional objectives are necessary, either due to the presence of strategic agents or due to fine-grained requirements. A central question we address is what kind of algorithm should be employed to play a repeated game against an unknown but rational opponent with consistent preferences? Although no-regret algorithms represent the standard solution from online learning and game theory, their guarantees are tight only against fully adversarial opponents. Against other opponents, i.e., in non-zero games, they can systematically underperform. We characterize this phenomenon by introducing a natural notion of optimality, Pareto-Optimality, for learning in games and show that popularly used no-regret algorithms such as multiplicative weights fail to satisfy this property. We then show that a strengthening of no-regret algorithms -- no-swap-regret algorithms -- satisfy this notion of optimality as well as another desirable property about being protected from strategic manipulation, non-manipulability. Then, we lift our results from normal form games to games with polytope action sets. In doing so, we solve a significant open problem in polytope games, by coming up with learning algorithms that are simultaneously efficiently implementable and have polynomial rates of convergence while being Pareto-Optimal and non-manipulable. These results are made possible by a novel framework that connects Blackwell Approachability with a geometric view of algorithms. We also use this framework to derive optimal commitment algorithms with and without the no-regret constraint in Stackelberg Equilibria for repeated games against a distribution of opponents. In the second half of the thesis, we explore applications of no-regret algorithms, and their strengthened versions achieved by using ideas from Blackwell approachability. We analyze two pivotal problems from a group fairness perspective: resource allocation for multistage interventions and online prediction. Our approach involves constructing optimal algorithms for these problems and extending them to incorporate group fairness properties. We also analyze algorithmic collusion through the lens of no-regret algorithms and best-responses to these algorithms. We derive the existence of algorithmic equilibria that induce collusive behavior without the usage of threats to enforce high prices.