Multiple influences on gaze and attention behavior for embodied agents

Erdan Gu, University of Pennsylvania


Our objective is to develop a computational model of multiple influences on eye gaze behavior for an embodied agent in a dynamic environment. An embodied agent should possess human attention attributes so that its eyes and resultant head movements convey appropriate and humanly understandable communicative behaviors. Suppressed or inappropriate eye movements can damage the visual effectiveness of an embodied agent. Visual attention models may be the key to leading animated agents out of the "uncanny valley" where increasing visual appearance accuracy results in a ghoulish appearance when animated. Visual perceptual capability, human-like imperfect cognitive ability, interpersonal interaction, and some aspects of internal cognitive state influence attention and eye gaze behaviors. Visual perceptual capability starts with an early vision process that exhibits changes in visual sensitivity such as night vision and flash blindness under changing scene illumination. Visual attention directs the limited gaze resource to resolve visual competition with the cooperation of top-down attention and conspicuous bottom-up guidance. Grounded in psychological studies, this model has four factors---conspicuity, mental workload, expectation and capacity---that determine successful attention allocation. The attention model replicates many aspects of normal human function as well as some of its imperfect behaviors, such as inattentional blindness. We implement and integrate attentional models into conversational agents in which gaze role is not only modeled based on the social aspects of interaction behaviors such as turn-taking and feedback signals, but also driven by peripheral events and abrupt onset. The attentional model is a separable component of an intelligent agent architecture, so that it may be transplanted to other applications, such as a virtual driver. Our contributions are a computational attention model, rational quantification of the inattention factors, integration into a general eye gaze model, application of the completed model to animated agents.

Subject Area

Computer science

Recommended Citation

Gu, Erdan, "Multiple influences on gaze and attention behavior for embodied agents" (2006). Dissertations available from ProQuest. AAI3246165.