|
pp. 5859-5874
S&M4656 Research Paper https://doi.org/10.18494/SAM6464 Published: October 20, 2026 Egocentric Industrial Human–Object Interaction Recognition Based on Hand Interaction Point Modeling [PDF] Yen-Chun Chen, Tsai-Yu Yao, and Po-Yu Chien (Received June 4, 2026; Accepted September 1, 2026) Keywords: egocentric vision, industrial human–object interaction recognition, hand interaction point modeling, hand segmentation, human–object interaction analysis
In intelligent manufacturing environments, first-person view imagery has become increasingly important for equipment maintenance, operation monitoring, and intelligent assistance systems. However, frequent hand occlusions, small interaction targets, and complex operational procedures make fine-grained human–object interaction (HOI) recognition challenging. In this paper, we propose an egocentric industrial HOI recognition method based on hand interaction point modeling, which integrates hand segmentation, object detection, a minimum-distance interaction search mechanism, and temporal analysis to establish hand–object associations and recognize interaction events. Experiments were conducted on the ENIGMA-51 industrial egocentric dataset and evaluated using the Egocentric HOI Detection task. Experimental results demonstrate that the proposed contour-based interaction point modeling significantly improves the HOI recognition performance, achieving 59.60% average precision for the hand and contact state (AP H.+State), 78.60% mean average precision for the hand and object state (mAP H.+Obj), and 45.63% mean average precision for the hand, object, and contact state (mAP H.+All), outperforming the conventional center-based interaction modeling approach. The results demonstrate that contour-based interaction modeling more accurately captures real hand–object contact relationships and provides more reliable interaction cues for industrial HOI recognition, offering an effective solution for intelligent factory operation comprehension and standard operating procedure analysis.
Corresponding author: Yen-Chun Chen![]() ![]() This work is licensed under a Creative Commons Attribution 4.0 International License. Cite this article Yen-Chun Chen, Tsai-Yu Yao, and Po-Yu Chien, Egocentric Industrial Human–Object Interaction Recognition Based on Hand Interaction Point Modeling, Sens. Mater., Vol. 38, No. 10, 2026, p. 5859-5874. |