pp. 3519-3537
S&M3748 Research Paper of Special Issue https://doi.org/10.18494/SAM4711 Published: August 29, 2024 Integration of Stacked Object Recognition and Robotic Arm Gripping System Utilizing Mask Region-based Convolutional Neural Network and Red–Green–Blue Depth Images [PDF] Shu-Yin Chiang and Yu-Kai Zhuo (Received October 26, 2023; Accepted May 14, 2024) Keywords: RGBD, Mask R-CNN, stacked object recognition, gripping angle, robotic arm gripping
In this study, we explore the challenge of object recognition by robots in scenarios where individual objects cannot be identified due to stacking. We harness the capabilities of the light detection and ranging (LiDAR) camera as a sensor for object detection, leveraging its advanced sensing technology to simultaneously capture both color and depth images of objects. To address the issue of image overlap caused by stacking, the Mask region-based convolutional neural network (R-CNN) is employed for object recognition and segmentation. Additionally, through image mapping transformation, the positions of individual objects in the red, green, and blue (RGB) image are projected onto the depth image to extract their corresponding depth information. Given the disparity in camera positions between the color and depth cameras, occlusions and variations in depth mapping can result in missing depth values for certain objects. To mitigate this, various statistical methods are utilized to fill in these missing values and enhance the accuracy of the extracted depth information. Furthermore, by calculating the width and length of the rectangle of the rotated image, the angle with the smallest value is selected as the gripping angle of the object. Finally, leveraging the transformed coordinates and planned trajectory, the robotic arm executes the gripping task on stacked objects. This execution process validates the accuracy of the three-dimensional spatial information and showcases the functionality of an intelligent robotic arm.
Corresponding author: Shu-Yin ChiangThis work is licensed under a Creative Commons Attribution 4.0 International License. Cite this article Shu-Yin Chiang and Yu-Kai Zhuo, Integration of Stacked Object Recognition and Robotic Arm Gripping System Utilizing Mask Region-based Convolutional Neural Network and Red–Green–Blue Depth Images, Sens. Mater., Vol. 36, No. 8, 2024, p. 3519-3537. |