A collaborative navigation algorithm for smart wheelchairs and AI glasses based on multimodal data fusion
Research Article
Open Access
CC BY

A collaborative navigation algorithm for smart wheelchairs and AI glasses based on multimodal data fusion

Zheng Yang 1* Jingfeng Yang 2, Xiyu Huang 3, Han Kong 4, Jiacheng Yan 5, Baoshan Wang 6, Rongcan Li 7, Huayang Cao 8
1 Geely University of China
2 Geely University of China
3 Geely University of China
4 Geely University of China
5 Geely University of China
6 Geely University of China
7 Geely University of China
8 Geely University of China
*Corresponding author: 267120970@qq.com
Published on 20 November 2025
Volume Cover
AORPM Vol.4 Issue 3
ISSN (Print): 3029-0899
ISSN (Online): 3029-0880
Download Cover

Abstract

In response to the growing demand for assistive devices under the backdrop of population aging, this paper proposes an innovative collaborative navigation algorithm integrating smart wheelchairs and AI glasses based on multimodal data fusion. The algorithm optimizes a closed-loop interaction among the “environment–user–device” triad. By integrating the wheelchair’s autonomous navigation capabilities with the AI glasses’ advanced environmental perception and real-time interaction functions, it significantly enhances system safety, autonomy, and user experience. The proposed approach employs a hybrid model combining Deep Belief Networks (DBN) and Stacked Autoencoders (SAE) to model and fuse multimodal data. Further integration with Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) enables improved system performance. Experimental comparisons in complex dynamic environments demonstrate that this method outperforms traditional Kalman filter–based fusion techniques. User survey results indicate a high level of acceptance and willingness to adopt this system among the target population. To promote its broader application, future research will focus on algorithm optimization, outdoor performance testing, and the enhancement of data security and privacy protection mechanisms.

Keywords:

smart wheelchair, AI glasses, multimodal data fusion, collaborative navigation, user interaction

View PDF
Yang,Z.;Yang,J.;Huang,X.;Kong,H.;Yan,J.;Wang,B.;Li,R.;Cao,H. (2025). A collaborative navigation algorithm for smart wheelchairs and AI glasses based on multimodal data fusion. Advances in Operation Research and Production Management,4(3),68-78.

References

[1]. Deligani, R. J., Hosni, S. I., & Borgheai, S. B. (2021). Multimodal EEG and fNIRS brain signal integration for enhanced neural decoding: A systematic review and bibliometric analysis. IEEE Transactions on Neural Systems and Rehabilitation Engineering, *29*, 43–43. https: //doi.org/10.1109/TNSRE.2021.3089975

[2]. Taborri, J., Keogh, J., & Kos, A. (2020). Sport biomechanics applications using inertial, force, and EMG sensors: A literature review.Applied Bionics and Biomechanics,2020, Article 2041549. https: //doi.org/10.1155/2020/2041549

[3]. Waisberg, E. (2024). Meta smart glasses—large language models and the future for assistive glasses for individuals with vision impairments.Eye, 38, 1036–1038. https: //doi.org/10.1016/S0140-6736(24)00111-2

[4]. Chignoli, M., Kim, D., & Stanger-Jones, E. (2020). The MIT humanoid robot: Design, motion planning, and control for acrobatic behaviors. 2020 IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids) (pp. 1–8). IEEE. https: //doi.org/10.1109/HUMANOIDS47582.2021.9555784

[5]. Fan, X. (2024). The communication pattern and industrial transformation in the era of mobile internet.China Media Technology, 3, 24–27.

[6]. Tao, L., Kui, Y., & Haibing, Z. (2008). Research status and development trend of Intelligent Wheelchair.Robot Technology and Application, 2, 1-5.

[7]. Han, S. (2008). Research on software adaptability oriented to mobility in ubiquitous computing [Doctoral dissertation, Shanghai Jiao Tong University].

[8]. Griffin, R. J. (2019). Footstep planning for autonomous walking over rough terrain. 2019 IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids) (pp. 9–16). IEEE. https: //doi.org/10.1109/HUMANOIDS47582.2021.9555795

[9]. Bandara, H. M. R. T. (2020). An intelligent gesture classification model for domestic wheelchair navigation with gesture variance compensation.Applied Bionics and Biomechanics, 2020, Article 9160528. https: //doi.org/10.1155/2020/9160528

[10]. Zhang, X., Li, J., & Zhang, R. (2024). A brain-controlled and user-centered intelligent wheelchair.Sensors, 24(10), 3201. https: //doi.org/10.3390/s24103201

[11]. Wensing, P. M., & Orin, D. E. (2016). Improved computation of the humanoid centroidal dynamics and application for whole-body control.International Journal of Humanoid Robotics, 13(1), 1550039. https: //doi.org/10.1142/S0219843615500398

[12]. Featherstone, R. (2014). Rigid body dynamics algorithms. Springer.

[13]. Lee, S. H., & Goswami, A. (2007). Reaction mass pendulum (RMP): An explicit model for centroidal angular momentum of humanoid robots. Proceedings of the 2007 IEEE International Conference on Robotics and Automation (pp. 4667–4672). IEEE. https: //doi.org/10.1109/ROBOT.2007.364191

[14]. Gao, J., Li, P., & Chen, Z. (2020). A survey on deep learning for multimodal data fusion.Neural Computation, 32(5), 829–864. https: //doi.org/10.1162/neco_a_01273

[15]. Katz, B., Di Carlo, J., & Kim, S. (2019). Mini cheetah: A platform for pushing the limits of dynamic quadruped control. 2019 International Conference on Robotics and Automation (ICRA) (pp. 6295–6301). IEEE. https: //doi.org/10.1109/ICRA.2019.8793865

[16]. Kim, D., Jorgensen, S. J., & Stone, P. (2016). Dynamic behaviors on the NAO robot with closed-loop whole body operational space control. 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids) (pp. 1121–1128). IEEE. https: //doi.org/10.1109/HUMANOIDS.2016.7803393

[17]. Sun, Z. (2022). Research on point cloud mapping and relocalization of indoor mobile robots [Master’s thesis, Southwest Jiaotong University].

[18]. Luo, K. (2022). A multi-robot mapping construction algorithm based on ORB-SLAM3.Experimental Technology and Management, 39(6), 82–91.

References

[1]. Deligani, R. J., Hosni, S. I., & Borgheai, S. B. (2021). Multimodal EEG and fNIRS brain signal integration for enhanced neural decoding: A systematic review and bibliometric analysis. IEEE Transactions on Neural Systems and Rehabilitation Engineering, *29*, 43–43. https: //doi.org/10.1109/TNSRE.2021.3089975

[2]. Taborri, J., Keogh, J., & Kos, A. (2020). Sport biomechanics applications using inertial, force, and EMG sensors: A literature review.Applied Bionics and Biomechanics,2020, Article 2041549. https: //doi.org/10.1155/2020/2041549

[3]. Waisberg, E. (2024). Meta smart glasses—large language models and the future for assistive glasses for individuals with vision impairments.Eye, 38, 1036–1038. https: //doi.org/10.1016/S0140-6736(24)00111-2

[4]. Chignoli, M., Kim, D., & Stanger-Jones, E. (2020). The MIT humanoid robot: Design, motion planning, and control for acrobatic behaviors. 2020 IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids) (pp. 1–8). IEEE. https: //doi.org/10.1109/HUMANOIDS47582.2021.9555784

[5]. Fan, X. (2024). The communication pattern and industrial transformation in the era of mobile internet.China Media Technology, 3, 24–27.

[6]. Tao, L., Kui, Y., & Haibing, Z. (2008). Research status and development trend of Intelligent Wheelchair.Robot Technology and Application, 2, 1-5.

[7]. Han, S. (2008). Research on software adaptability oriented to mobility in ubiquitous computing [Doctoral dissertation, Shanghai Jiao Tong University].

[8]. Griffin, R. J. (2019). Footstep planning for autonomous walking over rough terrain. 2019 IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids) (pp. 9–16). IEEE. https: //doi.org/10.1109/HUMANOIDS47582.2021.9555795

[9]. Bandara, H. M. R. T. (2020). An intelligent gesture classification model for domestic wheelchair navigation with gesture variance compensation.Applied Bionics and Biomechanics, 2020, Article 9160528. https: //doi.org/10.1155/2020/9160528

[10]. Zhang, X., Li, J., & Zhang, R. (2024). A brain-controlled and user-centered intelligent wheelchair.Sensors, 24(10), 3201. https: //doi.org/10.3390/s24103201

[11]. Wensing, P. M., & Orin, D. E. (2016). Improved computation of the humanoid centroidal dynamics and application for whole-body control.International Journal of Humanoid Robotics, 13(1), 1550039. https: //doi.org/10.1142/S0219843615500398

[12]. Featherstone, R. (2014). Rigid body dynamics algorithms. Springer.

[13]. Lee, S. H., & Goswami, A. (2007). Reaction mass pendulum (RMP): An explicit model for centroidal angular momentum of humanoid robots. Proceedings of the 2007 IEEE International Conference on Robotics and Automation (pp. 4667–4672). IEEE. https: //doi.org/10.1109/ROBOT.2007.364191

[14]. Gao, J., Li, P., & Chen, Z. (2020). A survey on deep learning for multimodal data fusion.Neural Computation, 32(5), 829–864. https: //doi.org/10.1162/neco_a_01273

[15]. Katz, B., Di Carlo, J., & Kim, S. (2019). Mini cheetah: A platform for pushing the limits of dynamic quadruped control. 2019 International Conference on Robotics and Automation (ICRA) (pp. 6295–6301). IEEE. https: //doi.org/10.1109/ICRA.2019.8793865

[16]. Kim, D., Jorgensen, S. J., & Stone, P. (2016). Dynamic behaviors on the NAO robot with closed-loop whole body operational space control. 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids) (pp. 1121–1128). IEEE. https: //doi.org/10.1109/HUMANOIDS.2016.7803393

[17]. Sun, Z. (2022). Research on point cloud mapping and relocalization of indoor mobile robots [Master’s thesis, Southwest Jiaotong University].

[18]. Luo, K. (2022). A multi-robot mapping construction algorithm based on ORB-SLAM3.Experimental Technology and Management, 39(6), 82–91.

Cite this article

Yang,Z.;Yang,J.;Huang,X.;Kong,H.;Yan,J.;Wang,B.;Li,R.;Cao,H. (2025). A collaborative navigation algorithm for smart wheelchairs and AI glasses based on multimodal data fusion. Advances in Operation Research and Production Management,4(3),68-78.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

About volume

Journal: Advances in Operation Research and Production Management

Volume number: Vol.4
Issue number: Issue 3
ISSN: 3029-0880(Print) / 3029-0899(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:

1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.

2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.

3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

For authors
For editors
For reviewers

Cookies are used by this site.
Copyright © 2024 EWA Publishing, its licensors, and contributors. All rights reserved, including text and data mining, AI training, and similar technologies.