Detecting anomalies in crowd movement is an area of considerable interest for surveillance and security applications. The question we address is: What constitutes an anomalous steering choice for an individual in the group? Deviation from “normal” behavior may be defined as a subject making a steering decision the observer would not, provided the same circumstances. Since the number of possible spatial and movement configurations is huge and human steering behavior is adaptive in nature, we adopt a context-sensitive approach to assess individuals rather than assume population-wide homogeneity. When presented with spatial trajectories from processed surveillance data, our system creates a shadow simulation. The simulation then establishes the current, local context for each agent and computes a predicted steering behavior against which the person’s actual motion can be statistically compared. We demonstrate the efficacy of our technique with preliminary results using real-world tracking data from the Edinburgh Pedestrian Dataset.