To become a native speaker, beyond obligatory rules, children need to learn systematic variation in the language, as it is present at all levels of language structure and is an integral part of linguistic knowledge. To give an example in English, speakers sometimes pronounce words ending in -ing with -in’ (e.g., working vs. workin’) depending on language-internal (e.g., -in’ is used more frequently in participles than in nouns like ‘morning’) and external (e.g., -in’ is used more frequently in casual conversational context than formal) factors. To learn such probabilistic patterns is a daunting challenge as children must figure out variants in alternation, factors modulating the alternation, relative rates associated with each variant in different contexts and more. Accounting for this process is crucial to any account of language acquisition, yet only a handful of studies have directly investigated children’s acquisition of variation. While emergent evidence suggests that children do acquire variation, we also know they tend to change their language input and make it more regular — a process often referred to as regularization. Taken together, a paradox arises: children apparently learn and match probabilistic linguistic variation under some circumstances but regularize such variation under others. When would children match variation and when would they regularize? While previous research identified many important factors contributing to this trade-off, including the presence of a conditioning factor, quantity of input etc., this question remains largely under-explored. Motivated by existing literature on creolization, developmental sociolinguistics and language acquisition, this dissertation investigates the impact of two well-grounded factors - the reliability of language input and shared variability across speakers - on how learners approach probabilistic variation. To do so, we leverage the artificial language learning paradigm: we expose adult and child learners to miniature languages containing variation and directly manipulate these factors one at a time to determine whether they provide them with important cues with respect to whether to regularize or learn the variation they are exposed to. We found that both child and adult learners were more likely to regularize variation when underlying regularity of the variation is not accessible to them (Chapter 3 & 4). While observing the same variable pattern shared by multiple speakers led adult learners to match the pattern more accurately (Chapter 5), children were not sensitive to this cue under the current paradigm (Chapter 6). Taken together, this dissertation shows that learners integrate cues beyond the statistical distribution of forms including structure of the variation, perceptual properties of the forms as well as how forms distribute in relation to speakers, to navigate probabilistic aspects of the language, and thus furthers our understanding on the cognitive mechanism that underlies the acquisition of variation in language.