The dual-route approach to speech perception proposes that a speaker’s voice activates perceived social attributes (e.g., age, gender, race) which are dynamically integrated with linguistic information during processing. How this integration unfolds over time as a listener reaches sentence-level interpretation remains unclear. To address this, we used eye-tracking to investigate when listeners integrate race-relevant cues in African American English (AAE) and Mainstream American English (MAE). In this study, 72 college-aged MAE-speaking participants viewed faces of two potential speakers (one African American, one European American) as they heard utterances that were (1) plausible both in MAE and AAE, (2) plausible AAE only, or (3) implausible both in MAE and AAE, and were asked to select the face of the most likely speaker. Because AAE is a stigmatized language variety, we expected that race-relevant cues would activate social attributes, impacting how participants differentially processed the utterances of the two speakers. Looking patterns and behavioral results show that participants quickly integrated race-relevant cues to associate the AAE-speaker with sentences not plausible in MAE. Surprisingly, within 900ms of hearing the AAE-speaker’s utterance unfold, participants were less likely to look at the African American face for plausible both sentences compared to the other sentence types. Similarly, participants were significantly slower and less likely to select the African American face as the speaker of plausible both sentences compared to the other sentence types. In contrast, when hearing the MAE-speaker, participants were significantly more likely to look at the European American face. A notable exception is 1000ms to 1200ms after plausible AAE sentence onset, when they looked significantly more to the African American face. However, participants quickly and accurately selected the European American face as the speaker regardless of sentence type. Despite participants’ apparent implicit knowledge of AAE, their performance was impacted by stigma surrounding this language variety and its speakers.