Object handover between human and robot

Description

Object handover is a common task in human life. It can be described as an interaction between two agents where one agent hands an object to the second agent. In the context of robotics, the agents can be human and robot, or two robots. This project aims to create a pipeline that allows smooth and safe object handover between a human and a robot (humanoid robot iCub or robotic arm) or between two robots. The intermediate steps consist of detecting and tracking objects in the scene, controlling the robot, and reacting to obstacles or other unpredictable situations. Thus, it requires a combination of robotics, computer vision, and deep learning (possibly with LLMs).

Requirements

1) Familiarize yourself with the literature and the robot 2) Create a pipeline that detects object of interest. Either in the hand of a human or on a table, etc. 3) Take the object:

• If the object is in the hand of a human, take it from the hand.
• If the objects is somewhere in the environment, grasp it. 4) Finish the handover:
• In case of human to robot handover, put the object on desired place.
• In case of robot to human handover, give the object to a human. 5) Signalize/visualiaze intent of the robot to the human in every task. 6) (Optional) Perform study with participants to test the method.

Literature

[1] Rosenberger, A. Cosgun, R. Newbury, J. Kwan, V. Ortenzi, P. Corke, and M. Grafinger, “Object-Independent Human-to-Robot Handovers Using Real Time Robotic Vision,” IEEE Robotics and Automation Letters, vol. 6, no. 1, pp. 17–23, Jan. 2021.

[2] P. D. Nguyen, F. Bottarel, U. Pattacini, M. Hoffmann, L. Natale, and G. Metta, “Merging Physical and Social Interaction for Effective Human-Robot Collaboration,” in 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), Nov. 2018, pp. 1–9.

[3] Y.-W. Chao, C. Paxton, Y. Xiang, W. Yang, B. Sundaralingam, T. Chen, A. Murali, M. Cakmak, and D. Fox, “HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers,” in 2022 International Conference on Robotics and Automation (ICRA), May 2022, pp. 6941–6947.