Multimodal Adaptive Generative AI Mechanism for Promoting L2 Oral Fluency Development

Ye Li; Yan Liang

doi:10.54254/2753-8818/2025.25631

Theoretical and Natural ScienceOpen access

Multimodal Adaptive Generative AI Mechanism for Promoting L2 Oral Fluency Development

Research Article

Open Access

Multimodal Adaptive Generative AI Mechanism for Promoting L2 Oral Fluency Development

Ye Li ^1* Yan Liang ²

¹ The Tourism College of Changchun University

² University of Edinburgh, Edinburgh, UK

^*Corresponding author: rara481846778@gmail.com

Published on 30 July 2025

TNS Vol.134

ISSN (Print): 2753-8826

ISSN (Online): 2753-8818

ISBN (Print): 978-1-80590-307-9

ISBN (Online): 978-1-80590-308-6

Download Cover

Abstract

Adaptive multimodal generation now enables artificial interlocutors that perceive speech, gaze, and gesture simultaneously and adjust feedback within milliseconds. Leveraging these advances, the present study engineers and validates a learner-adaptive system that fuses wav2vec-based speech recognition, a vision transformer for non-verbal cues, and a diffusion-avatar prompt engine trained through reinforcement learning with human fluency rubrics as reward. One hundred twenty intermediate English learners (B1–B2) practised with the agent or a teacher-led communicative syllabus for twelve weeks. Fine-grained telemetry captured 63 948 utterances, 5.7 million prosodic frames, and 173 hours of video frames. Mixed-effects growth modelling shows the AI group improved words-per-minute by 48.6 wpm (95 % CI = 42.4–54.8), mean-length-of-run by 3.91 syllables (CI = 3.34–4.48), and reduced filled-pause density by 6.3 pauses per 100 words (CI = 5.1–7.5), outperforming controls on all endpoints (p < 0.001). Learner diaries corroborate quantitative gains, citing lower anxiety and heightened prosodic experimentation. Findings evidence that synchronising cross-modal analytics with real-time generative feedback yields substantial fluency dividends and offer design principles for scalable AI-assisted speaking tutors.

Keywords:

multimodal learning, generative AI, adaptive feedback, oral fluency, second-language acquisition

View PDF

References

[1]. Tang, Zezong, and Yi Zhang. "The Potential Mechanisms and Approaches of Generative Artificial Intelligence in Oral English Education." 2024 4th International Conference on Educational Technology (ICET). IEEE, 2024.

[2]. He, Liqun, Manolis Mavrikis, and Mutlu Cukurova. "Designing and Evaluating Generative AI-Based Voice-Interaction Agents for Improving L2 Learners’ Oral Communication Competence." International Conference on Artificial Intelligence in Education. Cham: Springer Nature Switzerland, 2024.

[3]. Gaballo, Viviana. "Revolutionizing language teaching: AI in oral language assessment." Conference Proceedings. Innovation in Language Learning 2024. 2024.

[4]. Zapata, Gabriela C., ed. Generative AI Technologies, Multiliteracies, and Language Education. Taylor & Francis, 2025.

[5]. Li, Mengdi, Yinyu Wang, and Xiaorong Yang. "Can Generative AI Chatbots Promote Second Language Acquisition? A Meta‐Analysis." Journal of Computer Assisted Learning 41.4 (2025): e70060.

[6]. Godwin-Jones, Robert. "Distributed agency in second language learning and teaching through generative AI." arXiv preprint arXiv: 2403.20216 (2024).

[7]. Tzirides, Anastasia Olga Olnancy. Smart online language learning modules: An exploration of the potential of advanced digital technologies and artificial intelligence for collaborative language learning utilizing translanguaging and multimodal communication approaches. Diss. University of Illinois at Urbana-Champaign, 2022.

[8]. Ironsi, Chinaza Solomon. "Exploring the potential of generative AI in English language teaching." Facilitating global collaboration and knowledge sharing in higher education with generative AI. IGI Global Scientific Publishing, 2024. 162-185.

[9]. Nanduri, Dinesh Kumar. Exploring the Role of Generative Artificial Intelligence in Culturally Relevant Storytelling for Native Language Learning Among Children. MS thesis. University of Maryland, College Park, 2024.

References

[3]. Gaballo, Viviana. "Revolutionizing language teaching: AI in oral language assessment." Conference Proceedings. Innovation in Language Learning 2024. 2024.

[4]. Zapata, Gabriela C., ed. Generative AI Technologies, Multiliteracies, and Language Education. Taylor & Francis, 2025.

[5]. Li, Mengdi, Yinyu Wang, and Xiaorong Yang. "Can Generative AI Chatbots Promote Second Language Acquisition? A Meta‐Analysis." Journal of Computer Assisted Learning 41.4 (2025): e70060.

[6]. Godwin-Jones, Robert. "Distributed agency in second language learning and teaching through generative AI." arXiv preprint arXiv: 2403.20216 (2024).

Cite this article

Li,Y.;Liang,Y. (2025). Multimodal Adaptive Generative AI Mechanism for Promoting L2 Oral Fluency Development. Theoretical and Natural Science,134,14-19.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

About volume

Volume title: The 3rd International Conference on Applied Physics and Mathematical Modeling

ISBN: 978-1-80590-307-9(Print) / 978-1-80590-308-6(Online)

Editor: Marwan Omar

Conference website: https://2025.confapmm.org/

Conference date: 31 October 2025

Series: Theoretical and Natural Science

Volume number: Vol.134

ISSN: 2753-8818(Print) / 2753-8826(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:

1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.

2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.

3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).