This dissertation investigates people’s ability to search for and use alternative reasons while making probabilistic judgments, with the goals of devising and testing (1) a new actively open-minded thinking (AOT) measure that assesses thinking behavior by looking at people’s ability to generate alternative reasons/contradicting evidence, and (2) a short online module to train people in actively open-minded thinking. In Chapter 1 we assessed individual differences in actively open-minded thinking on probabilistic judgment tasks by using both belief and behavioral measures. In the first three studies subjects made probabilistic judgments in three-choice almanac questions, while in the fourth study subjects made point and confidence interval estimates for numerical almanac questions. Compared to the low scoring subjects, subjects who score high on the new behavioral AOT measure were more likely to have more accurate probability judgments when they did not know the correct answer to the question. Higher scores on the behavioral actively open-minded thinking measure were also associated with lower overconfidence. In Chapter 2 we tested the effectiveness of making subjects consider alternative reasons and a one-hour long online training module in AOT. Studies 4 and 5 tested whether making subjects consider alternative reasons would improve their accuracy and decrease their overconfidence. In Study 4 we observed that this intervention was successful in increasing the number of alternative reasons and subjects’ accuracy when subjects did not know the correct answer. There was also a slight decrease in overconfidence as a result of this intervention. Study 5, which used point estimate questions, did not show any benefits of the intervention in accuracy or overconfidence. Studies 6 and 7 tested the effectiveness of new online AOT training modules we designed for adults. The training module in Study 6 did not show any improvement in subjects’ accuracy, but the training condition in Study 7 increased subjects’ accuracy scores when subjects did not know the correct answer. We observed some effect of the training on subjects’ overconfidence such that going through the training decreased subjects’ unwarranted confidence. Chapter 3 discusses the relation between the behavioral and belief measures of AOT, their effects on accuracy and overconfidence. We specifically argue that while the belief measure of AOT assesses general AOT tendencies, the behavioral measure of AOT assesses task specific AOT behavior. The results show that considering alternative reasons increases subjects’ accuracy by lowering their overconfidence. We finally discuss our results from training adult population in AOT and suggest potential testing scenarios for our training module.