Human communication relies on shared expectations between speakers and hearers. For example, upon hearing a sentence like “Some of my dogs bark,” the listener typically assumes that the speaker did not intend the literal semantic meaning (“At least one (and possibly all) of my dogs bark”). Instead, s/he is likely to derive a scalar implicature (SI), inferring that the speaker intended to convey “Not all of my dogs bark.” Properties of the speaker are known to affect whether listeners compute SIs, with comprehenders being less likely to make a pragmatic inference when the speaker is not knowledgeable of the situation at hand. What is unclear is whether listeners also use previously-held expectations about speaker groups (e.g., children, non-native speakers) to override Gricean principles, in such a way that is stable across situational contexts and does not require an adaptation period. Across two experiments, we investigated how listeners interpret under-informative utterances produced by native and non-native speakers. We found that a subset of individuals is more tolerant to pragmatic infelicities produced by non-native speakers (Exp. 2), but that this tolerance is subject to individual differences in language processing ability and does not emerge in a speeded task where the utterances are supported by visual context (Exp. 1).