Manipulator Path Planning for Multi-Object Grasping in a Declutter Problem

Abstract

The declutter problem is a fundamental pick-and-place task in domestic and industrial environments, aiming to remove all objects from the workspace. Conventional approaches focus on motion planning to avoid obstacles rather than minimizing process time, as they typically use grippers that handle one object at a time. Recent studies have investigated multi-object grasping, utilizing parallel grippers to collect and simultaneously transport multiple objects from a table for efficient pick-and-place. However, these studies have limitations regarding the combinations of objects' orientations that can be stably grasped, as contact occurs between objects when grasping multiple items at once. Consequently, conventional approaches prioritize grasp planning over manipulator path optimization for efficient decluttering. Here, we propose an efficient decluttering approach that incorporates multi-object grasping with a palm design enabling simultaneous storage and a manipulator path planning method to optimize the process. The conveyor palm, featuring opposing belts with embedded elastic hairs, enables stable multi-object storage without contact between objects, accommodating various orientations. Additionally, we introduce TSP_Store, an extension of the Traveling Salesman Problem (TSP), as a manipulator path planning method for multi-object grasping that considers finite storage capacity in path optimization. Experimental comparisons show that TSP_Store reduces manipulator’s travel distance by 15.9% and process time by 10.3% compared to conventional TSP. Our work highlights the effectiveness of TSP-based path planning for multi-object grasping and emphasizes the necessity of incorporating storage limits into the planning process.

Video

This video shows demonstration of developed path planning algorithm.

Declutter Problem Using MOGrip [1]

The proposed MOGrip is less affected by the order of objects when storing them. As a result, it overcomes the limitation of previous studies on multi-object grasping for decluttering, where the combination of objects that could be grasped at once was limited [2]. This allows for path planning of manipulator in declutter problem, rather than being constrained by grasp planning alone.

For decluttering, MOGrip moves to the locations of the objects and collects them one by one. The objective of path planning is to find the path that minimizes the manipulator’s travel distance or time. This problem is similar to the well-known Traveling Salesman Problem (TSP).

Differences Between Path Planning for Multi-Object Grasping and Conventional TSP

The key difference between the Traveling Salesman Problem (TSP) and multi-object grasping using MOGrip is the gripper’s limited storage capacity. When following the path obtained by solving TSP, the storage eventually becomes full, requiring the manipulator to perform additional round trips. As a result, the actual path becomes longer than the one derived from TSP (green lines in the figure).

By incorporating the limited storage capacity into path planning, a shorter manipulator path can be obtained (blue lines in the figure). Accordingly, we propose TSP_Store, a method that solves TSP while considering the gripper’s storage constraints.

Storage Capacity

To develop TSP_Store, it is necessary to define the storage capacity of the proposed gripper. Whether the gripper can store all given objects depends on the space the objects occupy in the conveyor palm upon storage, which we define as occupied space. The occupied space is defined as the minimum space required that does not interfere with the storage of other objects. The size of the occupied space varies depending on the object’s diameter.

Path Planning Pipeline

For path planning, the object setting was captured by camera, and their positions and sizes were calculated through segmentation. Based on these calculated values, TSP_Store is utilized to find the minimum path.

Results

We compared the travel distance and decluttering process time of single-object grasping (SOG), multi-object grasping along the path obtained by TSP, and multi-object grasping along the path generated by TSP_Store. In terms of travel distance, the path generated using the TSP_Store method was 65.7% shorter than that of SOG and 15.9% shorter than that of TSP. Additionally, for the decluttering process time, the path generated using the TSP_Store method was 40.5% more efficient than SOG and 10.3% more efficient than TSP.

📌 Reference

[1] Jaemin Eom et al., “MOGrip: Gripper for multiobject grasping in pick-and-place tasks using translational movements of fingers,” Science Robotics, 9, eado3939(2024).

[2] W. C. Agboh et al., “Learning to Efficiently Plan Robust Frictional Multi-Object Grasps,” 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA, 2023, pp. 10660-10667.