Many recent neuroimaging studies have investigated the representation of semantic memory for actions in the brain. We used activation likelihood estimation (ALE) meta-analyses to answer two outstanding questions about the neural basis of action concepts. First, on an “embodied” view of semantic memory, evidence to date is unclear regarding whether visual motion or motor systems are more consistently engaged by action concepts. Second, few studies have directly investigated the possibility that action concepts accessed verbally or nonverbally recruit different areas of the brain. Because our meta-analyses did not include studies requiring the perception of dynamic depictions of actions or action execution, we were able to determine whether conceptual processing alone recruits visual motion and motor systems. Significant concordance in brain regions within or adjacent to visual motion areas emerged in all meta-analyses. By contrast, we did not observe significant concordance in motor or premotor cortices in any analysis. Neural differences between action images and action verbs followed a gradient of abstraction among representations derived from visual motion information in the left lateral temporal and occipital cortex. The consistent involvement of visual motion but not motor brain regions in representing action concepts may reflect differences in the variability of experience across individuals with perceiving versus performing actions.