|
pp. 5847-5858
S&M4655 Research Paper https://doi.org/10.18494/SAM6463 Published: October 20, 2026 Large Language Model-based Textualization of Gradient-weighted Class Activation Mapping Explanations for Osteoporosis Prediction from Sensor-acquired Hip Radiographs [PDF] Shang-Wen Feng, Hao-Chun Chiang, Yu-Sheng Huang, Chen-Ni Hu, Yi-Hung Chiang, and Szu-Yin Lin (Received June 4, 2026; Accepted October 13, 2026) Keywords: osteoporosis prediction, sensor-acquired hip radiographs, explainableAI, Grad-CAM, large language models, prompt engineering
Osteoporosis screening from hip radiographs requires both predictive performance and interpretable model outputs. In this study, we developed a three-component computational framework that separates binary image classification from post hoc visual and textual explanation. Four ImageNet-pretrained models, namely, VGG16, ResNet50, InceptionV3, and ViT-B16, were evaluated using patient-level fivefold cross-validation. VGG16 achieved the highest observed aggregate accuracy of 0.80 and was selected for subsequent explainability analysis. Gradient-weighted class activation mapping (Grad-CAM) was then used to localize image regions associated with the classifier output, and ChatGPT-4o converted the activation patterns and level-specific contextual information into structured textual explanations under five prompt-engineering conditions. The five prompt-engineering conditions progressively incorporated symbolic representation, anatomical context, structured reasoning, and critical clinical review. In the representative case, Grad-CAM activation was mainly observed around the femoral neck, proximal femur, and adjacent cortical regions. The generated outputs illustrate how prompt design can alter the detail, organization, and clinical framing of textualized explainable artificial intelligence (XAI) explanations. These findings demonstrate the technical feasibility of using a large language model (LLM) as a post hoc textualization layer for sensor-acquired hip radiographs, although further multicase and clinician-based validation is required.
Corresponding author: Szu-Yin Lin![]() ![]() This work is licensed under a Creative Commons Attribution 4.0 International License. Cite this article Shang-Wen Feng, Hao-Chun Chiang, Yu-Sheng Huang, Chen-Ni Hu, Yi-Hung Chiang, and Szu-Yin Lin, Large Language Model-based Textualization of Gradient-weighted Class Activation Mapping Explanations for Osteoporosis Prediction from Sensor-acquired Hip Radiographs , Sens. Mater., Vol. 38, No. 10, 2026, p. 5847-5858. |