This article focuses on challenges to improving the realism of socially intelligent agents and attempts to reflect the state of the art in human behavior modeling with particular attention to the impact of personality/cultural values and affect as well as biology/stress upon individual coping and group decision-making. The first section offers an assessment of the state of the practice and of the need to integrate valid human performance moderator functions (PMFs) from traditionally separated sub-fields of the behavioral literature. The second section pursues this goal by postulating a unifying architecture and principles for integrating existing PMF theories and models. It also illustrates a PMF testbed called PMFserv created for implementating and studying how PMFs may contribute to such an architecture. To date it interconnects versions of PMFs on physiology and stress (Janis-Mann, Gillis-Hursh, others); personality, cultural and emotive processes (Damasio, Cognitive Appraisal-OCC, value systems); perception (Gibsonian affordance); social processes (relations, identity, trust, nested intentionality); and cognition (affect- and stress-augmented decision theory, bounded rationality). The third section summarizes several usage case studies (asymmetric warfare, civil unrest, and political leaders) and concludes with lessons learned. Implementing and inter-operating this broad collection of PMFs helps to open the agenda for research on syntheses that can help the field reach a greater level of maturity. Part II presents a case study in using PMFserv for rapid scenario composability and realistic agent behavior.