Decision making has been studied extensively throughout the long history of both economics and operations management due to its importance for decision makers. But it is not until recently when substantial decision-related data become available that decision-making algorithm designs start to rely heavily on machine learning techniques. Indeed, the rapid growth in data has driven successful machine learning solutions that can inform decision making across industries. However, decision makers also face many new challenges in practice, such as effective decision making in the presence of data scarcity as well as complex model structure such as neural network. To address these practical concerns, I design data-driven algorithms for both sequential and static decision making that are tailored to these challenges, and provide fundamental characterizations of the performance of these methods. My work has been motivated by these kind of practical problems that arise in a wide variety of application domains, including healthcare operations, dynamic pricing and textual analytics. Specifically, this dissertation provides novel solutions that encourage collaboration and embrace knowledge sharing among multiple decision makers with limited data resources. We carefully design the model structure that accounts for the heterogeneity across decision makers and the contextual complexity for each decision maker. In addition, we provide both theoretical and empirical justifications that quantitatively evaluate the performance of these algorithms, and demonstrate the advantages of these proposed algorithms in example applications.