Technical Reports (CIS)

Document Type

Technical Report

Date of this Version



University of Pennsylvania Department of Computer and Information Science Technical Report No. MS-CIS-11-09.


We introduce a method for tracking multiple people in a
cluttered street scene. We use global context to address the challenge of
long occlusion by endowing each tracked object with a planning agent.
This planner uses context of the street scene, people and other moving
objects to reason about pedestrian intended behavior for tracking under
occlusion and ambiguity.

We extract short but robust trajectories called tracklets by tracking people
with a simple appearance model. We formulate the tracking problem
as a batch mode optimization, linking tracklets into paths, each with
supporting evidence by an agent’s goal directed behavior, and image
partial matching along the trajectory gap. We propose a global criteria
for consistent linking of the tracklet with planning that can correct local
ambiguity in linking. We test our algorithm in a challenging real world
setting, where we automatically estimate scene context and intended
goals, then track multiple people from a moving camera.



Date Posted: 14 April 2011