This paper focuses on the grammaticality of ‘by’-phrases in the Icelandic New Impersonal Passive (NIP), a construction that exhibits both passive morphology and properties that are characteristic of the active. The analysis of the NIP is debated, including whether or not ‘by’-phrases are grammatical, which is one of the most important questions for the syntactic analysis of the NIP. In the paper, we focus on this particular question and compare the NIP to the Low Canonical Passive (LCP). We review two judgment tasks in which the NIP was extensively studied, one conducted by Joan Maling and Sigríður Sigurjónsdóttir and another conducted in the Variation in Syntax project at the University of Iceland. As Jónsson (2009) argues, the results indicate that ‘by’-phrases are indeed grammatical in the NIP but the results are not clear-cut. To gain a better understanding of the problem, we conducted a different kind of a study among 12-16-year-olds (n=309) and adults (n=38). We used fragment answers to the question ‘What language is spoken in Norway?’ to categorize NIP speakers (the construction is ungrammatical for many speakers) and Other speakers. The wh-phrase is syncretic for nominative and accusative and here a Canonical Passive participle is identical to a NIP participle. The answer (Norwegian) is not syncretic for both cases and therefore reveals the speaker’s analysis of the question. We compared these results to how speakers answered a similar question that contained a ‘by’-phrase, ‘What language is spoken by Finns?’ If the NIP were an active construction, we would not expect any speakers to produce accusative when answering the latter question. Still, a few speakers did, indicating that ‘by’-phrases are grammatical in the NIP. Accusative was, however, produced significantly more often when speakers answered the first question. We also tested a) how NIP speakers understood a NIP sentence with a PP that should be ambiguous between meaning ‘by’ and ‘from’, and b) how LCP speakers understand an identical LCP sentence. Our results suggest that ‘by’-phrases are grammatical in the NIP but the possibility remains that they are only grammatical for some NIP speakers but ungrammatical for other NIP speakers.