Departmental Papers (ESE)


Streaming Media


Complex manipulation tasks, such as rearrangement planning of numerous objects, are combinatorially hard problems. Existing algorithms either do not scale well or assume a great deal of prior knowledge about the environment, and few offer any rigorous guarantees. In this paper, we propose a novel hybrid control architecture for achieving such tasks with mobile manipulators. On the discrete side, we enrich a temporal logic specification with mobile manipulation primitives such as moving to a point, and grasping or moving an object. Such specifications are translated to an automaton representation, which orchestrates the physical grounding of the task to mobility or manipulation controllers. The grounding from the discrete to the continuous reactive controller is online and can respond to the discovery of unknown obstacles or decide to push out of the way movable objects that prohibit task accomplishment. Despite the problem complexity, we prove that, under specific conditions, our architecture enjoys provable completeness on the discrete side, provable termination on the continuous side, and avoids all obstacles in the environment. Simulations illustrate the efficiency of our architecture that can handle tasks of increased complexity while also responding to unknown obstacles or unanticipated adverse configurations.

For more information: Kod*lab

Sponsor Acknowledgements

This work was supported by AFRL grant FA865015D1845 (subcontract 669737-1), AFOSR grant FA9550-19-1-0265 (Assured Autonomy in Contested Environments), and ONR grant #N00014-16-1-2817, a Vannevar Bush Fellowship held by the last author, sponsored by the Basic Research Office of the Assistant Secretary of Defense for Research and Engineering.

Document Type

Conference Paper

Subject Area

GRASP, Kodlab

Date of this Version


Publication Source

IEEE International Conference on Robotics and Automation (ICRA)

Copyright/Permission Statement

This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of the University of Pennsylvania's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to By choosing to view this document, you agree to all provisions of the copyright laws protecting it.


Reactive and Sensor-Based Planning, Task and Motion Planning, Formal Methods in Robotics and Automation

Bib Tex

@InProceedings{vasilopoulos_kantaros_pappas_koditschek_2021, Title = {{Reactive Planning for Mobile Manipulation Tasks in Unexplored Semantic Environments}}, Author = {V. Vasilopoulos and Y. Kantaros and G. J. Pappas and D. E. Koditschek}, Booktitle = {IEEE International Conference on Robotics and Automation}, Year = {2021}}



Date Posted: 05 April 2021

This document has been peer reviewed.