|
pp. 5243-5258
S&M4621 Technical Paper https://doi.org/10.18494/SAM6478 Published: September 18, 2026 Applying Transformers and Knowledge Graphs to Home Care Event Detection [PDF] Jui-Feng Yeh, Chun-Yu Tsai, Bo-Sung Huang, Po-Hsiang Yang, and Chen-Yu Yeh (Received June 9, 2026; Accepted September 9, 2026) Keywords: home care, transformer, knowledge graph, privacy protection, virtual avatar, digital twin
To address the home-care problems resulting from an increasing number of elderly people living alone and to prevent related accidents, we develop a multimodal event detection system that combines knowledge graphs and transformer architecture for monitoring elderly people by sensor technologies. This system focuses on analyzing behavioral events of older adults in home care settings. In consideration of privacy, a digital twin is adopted in the monitoring video, replacing the original content with virtual avatars. There are mainly two phrases in the proposed system framework. First, the proposed system utilizes the Knowledge-Enabled Language Representation Model enabling language representation with knowledge graph as its core, integrating semantic features from knowledge graphs, and enhancing language understanding through the self-attention mechanism of the transformer. Secondly, the Bootstrapping Language-Image Pre-training model is used for video narration generation, combined with the GroundingDINO model for object detection to enable the narration to correspond to objects in the actual scene. Finally, a text report is output and presented in a visual player. It can also be used as input for AI models to analyze behavioral events in virtual home care spaces. The experimental results indicate that after introducing generative AI, the recall rate for running events increased significantly to 83.3%. These results confirm that the system can effectively improve the accuracy of hazard detection in home care.
Corresponding author: Jui-Feng Yeh![]() ![]() This work is licensed under a Creative Commons Attribution 4.0 International License. Cite this article Jui-Feng Yeh, Chun-Yu Tsai, Bo-Sung Huang, Po-Hsiang Yang, and Chen-Yu Yeh, Applying Transformers and Knowledge Graphs to Home Care Event Detection, Sens. Mater., Vol. 38, No. 9, 2026, p. 5243-5258. |