Digital games with realistic virtual characters have become very popular. The ability for players to promptly control their character is a crucial feature of these types of games, be it platform games, first-person shooters, or role-playing games. Controller latencies, meaning delays in the responsiveness of a player’s character, for example due to extensive computations or to network latencies, can considerably reduce the player’s enjoyment of a game. In this paper, we present a thorough analysis of the consequences of such delays on the player’s experience across three parts of a game with different levels of difficulty. We investigate the effects of responsiveness on the player’s enjoyment, performance, and perception of the game, as well as the player’s adaptability to delays. We find that responsiveness is very important for the player as delays affect the player’s enjoyment of the game as well as the player’s performance. A quick responsiveness becomes essential for more challenging tasks.