Science is an essential subject in students’ daily life and their future careers. The factors that affect science achievement across countries have been widely discussed in comparative education literature. Moreover, there is little systematic understanding about whether the effects of a wide scope of factors on science achievement in one country are generalizable to other countries of similar features and/or whether there is alignment among the different measures of generalizability. This dissertation aims to fill these research gaps by exploring a wide collection of factors at both the student- and the school-level that are associated with science achievement, with a focus on students from disadvantaged SES backgrounds in 30 economies that participated in the Programme for International Student Assessment (PISA) 2015 wave. This dissertation assesses the extent to which the factors that affect student achievement in science generalize across multiple contexts, which include country groups, variable types and SES backgrounds. Reviewing and summarizing the different definitions of generalizability in practice, a goal of this dissertation is to examine the extent to which there is alignment among the results under each definition and the implications of this alignment (or misalignment) for researchers and policymakers. Findings support that the measures of generalizability discussed in this dissertation did not align with each other in most cases, potentially because of their distinct assumptions, goals and implications.