Semantic Segmentation and Object Detection for 3D Motion Analysis of the Ankle Joint in High-Resolution MRI
Research Article
Open Access
CC BY

Semantic Segmentation and Object Detection for 3D Motion Analysis of the Ankle Joint in High-Resolution MRI

Ruizhe Liu 1*
1 New York University, Brooklyn, New York State, United States, 11217
*Corresponding author: rachelliucqt@gmail.com
Published on 30 July 2025
Volume Cover
ACE Vol.175
ISSN (Print): 2755-273X
ISSN (Online): 2755-2721
ISBN (Print): 978-1-80590-237-9
ISBN (Online): 978-1-80590-238-6
Download Cover

Abstract

Although high-resolution MRI provides excellent anatomical detail, existing segmentation approaches possess a requisite yet inadequate level of precision, require substantial human effort, and fail to accurately represent the intricate 3D structure. To address these limitations, this work develops a novel 3D Faster R-CNN engine that automatically detects and segments the main ankle joint components from volumetric MRI. The proposed design combines a 3D ResNet-50 transformer with a 3D Region Proposal Network and 3D ROI Align components to analyze MRI scans. The model trained with experiments based on ankle MRI datasets from second-party repositories used data processing steps to normalize image size and enhance dataset collection. The assessment metrics consisted of Dice Similarity Coefficient, Intersection over Union, and mean Average Precision (mAP). By evaluating several models, the system achieves a Dice coefficient score of 91.4% alongside an mAP of 89.6% at IoU 0.5 which beats previous 2D and 3D segmentation techniques. Scientific images showed that the method could precisely detect body structures in different MRI views while keeping their correct shapes.

Keywords:

Object detection, Faster region based convolutional neural network (Faster-RCNN), 3D motion, Ankle joint, Recognition

View PDF
Liu,R. (2025). Semantic Segmentation and Object Detection for 3D Motion Analysis of the Ankle Joint in High-Resolution MRI. Applied and Computational Engineering,175,42-50.

References

[1]. Blemker, S. S., Asakawa, D. S., Gold, G. E., & Delp, S. L. (2007). Image-based musculoskeletal modeling: Applications, advances, and future opportunities. Journal of Magnetic Resonance Imaging, 25(2), 441–451.

[2]. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, 9351, 234–241.

[3]. Çiçek, Ö., Abdulkadir et al. (2016). 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. Medical Image Computing and Computer-Assisted Intervention ; 9901, 424–432.

[4]. Litjens, G., Kooi, T., Bejnordi, B. E., et al. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60–88. https: //doi.org/10.1016/j.media.2017.07.005mRedmon, J., & Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv preprint arXiv: 1804.02767.

[5]. Wang, L., et al. (2019). 3D Segmentation of Ankle Bones From Weight-Bearing CT Images Using a Hierarchical CNN. IEEE Transactions on Medical Imaging, 38(6), 1337–1347. https: //doi.org/10.1109/TMI.2018.2889433

[6]. Gold, G. E., et al. (2007). Musculoskeletal MRI at 3.0 T: Relaxation times and image contrast. American Journal of Roentgenology, 188(2), 343–351. https: //doi.org/10.2214/AJR.06.0404

[7]. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., & Zhou, Y. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv preprint arXiv: 2102.04306.

[8]. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI 2015, pp. 234–241. https: //doi.org/10.1007/978-3-319-24574-4_28

[9]. Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T., & Ronneberger, O. (2016). 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. MICCAI 2016, pp. 424–432.

[10]. Milletari, F., Navab, N., & Ahmadi, S. A. (2016). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. Proceedings of 3DV, pp. 565–571. https: //doi.org/10.1109/3DV.2016.79

[11]. Wang, L., et al. (2019). 3D Segmentation of Ankle Bones from Weight-Bearing CT Images Using a Hierarchical CNN. IEEE Transactions on Medical Imaging, 38(6), 1337–1347. https: //doi.org/10.1109/TMI.2018.2889433

[12]. Oktay, O., Schlemper, J., Le Folgoc, L., et al. (2018). Attention U-Net: Learning Where to Look for the Pancreas. arXiv preprint arXiv: 1804.03999.

[13]. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., & Zhou, Y. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv preprint arXiv: 2102.04306.

[14]. Zhou, Z., et al. (2018). UNet++: A Nested U-Net Architecture for Medical Image Segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, 3–11.

[15]. Li, X., Chen et al. (2020). H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes. IEEE Transactions on Medical Imaging, 37(12), 2663–2674.

[16]. Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., Han, X., & Wu, J. (2019). 3D U²-Net: A 3D Universal U-Net for Multi-domain Medical Image Segmentation. arXiv preprint arXiv: 1909.10371.

[17]. Osteoarthritis Initiative (OAI). (2022). OAI MRI Dataset: High-resolution 3D MR Images of the Musculoskeletal System (including ankle and lower limb scans) [Data set]. National Institutes of Health. https: //nda.nih.gov/oai/

References

[1]. Blemker, S. S., Asakawa, D. S., Gold, G. E., & Delp, S. L. (2007). Image-based musculoskeletal modeling: Applications, advances, and future opportunities. Journal of Magnetic Resonance Imaging, 25(2), 441–451.

[2]. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, 9351, 234–241.

[3]. Çiçek, Ö., Abdulkadir et al. (2016). 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. Medical Image Computing and Computer-Assisted Intervention ; 9901, 424–432.

[4]. Litjens, G., Kooi, T., Bejnordi, B. E., et al. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60–88. https: //doi.org/10.1016/j.media.2017.07.005mRedmon, J., & Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv preprint arXiv: 1804.02767.

[5]. Wang, L., et al. (2019). 3D Segmentation of Ankle Bones From Weight-Bearing CT Images Using a Hierarchical CNN. IEEE Transactions on Medical Imaging, 38(6), 1337–1347. https: //doi.org/10.1109/TMI.2018.2889433

[6]. Gold, G. E., et al. (2007). Musculoskeletal MRI at 3.0 T: Relaxation times and image contrast. American Journal of Roentgenology, 188(2), 343–351. https: //doi.org/10.2214/AJR.06.0404

[7]. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., & Zhou, Y. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv preprint arXiv: 2102.04306.

[8]. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI 2015, pp. 234–241. https: //doi.org/10.1007/978-3-319-24574-4_28

[9]. Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T., & Ronneberger, O. (2016). 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. MICCAI 2016, pp. 424–432.

[10]. Milletari, F., Navab, N., & Ahmadi, S. A. (2016). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. Proceedings of 3DV, pp. 565–571. https: //doi.org/10.1109/3DV.2016.79

[11]. Wang, L., et al. (2019). 3D Segmentation of Ankle Bones from Weight-Bearing CT Images Using a Hierarchical CNN. IEEE Transactions on Medical Imaging, 38(6), 1337–1347. https: //doi.org/10.1109/TMI.2018.2889433

[12]. Oktay, O., Schlemper, J., Le Folgoc, L., et al. (2018). Attention U-Net: Learning Where to Look for the Pancreas. arXiv preprint arXiv: 1804.03999.

[13]. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., & Zhou, Y. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv preprint arXiv: 2102.04306.

[14]. Zhou, Z., et al. (2018). UNet++: A Nested U-Net Architecture for Medical Image Segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, 3–11.

[15]. Li, X., Chen et al. (2020). H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes. IEEE Transactions on Medical Imaging, 37(12), 2663–2674.

[16]. Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., Han, X., & Wu, J. (2019). 3D U²-Net: A 3D Universal U-Net for Multi-domain Medical Image Segmentation. arXiv preprint arXiv: 1909.10371.

[17]. Osteoarthritis Initiative (OAI). (2022). OAI MRI Dataset: High-resolution 3D MR Images of the Musculoskeletal System (including ankle and lower limb scans) [Data set]. National Institutes of Health. https: //nda.nih.gov/oai/

Cite this article

Liu,R. (2025). Semantic Segmentation and Object Detection for 3D Motion Analysis of the Ankle Joint in High-Resolution MRI. Applied and Computational Engineering,175,42-50.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

About volume

Volume title: Proceedings of CONF-CDS 2025 Symposium: Application of Machine Learning in Engineering

ISBN: 978-1-80590-237-9(Print) / 978-1-80590-238-6(Online)
Editor: Marwan Omar, Mian Umer Shafiq
Conference website: https://www.confcds.org
Conference date: 19 August 2025
Series: Applied and Computational Engineering
Volume number: Vol.175
ISSN: 2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:

1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.

2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.

3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

For authors
For editors
For reviewers

Cookies are used by this site.
Copyright © 2024 EWA Publishing, its licensors, and contributors. All rights reserved, including text and data mining, AI training, and similar technologies.