LoRA fine-tuned Qwen2.5-VL large model for accurate description and location of steel surface defects

Jingliang Liu

doi:10.54254/2977-3903/2025.26796

Advances in Engineering InnovationOpen access

LoRA fine-tuned Qwen2.5-VL large model for accurate description and location of steel surface defects

Research Article

Open Access

LoRA fine-tuned Qwen2.5-VL large model for accurate description and location of steel surface defects

Jingliang Liu ^1*

¹ University of Electronic Science and Technology of China

^*Corresponding author: JayLiu090@163.com

Published on 8 September 2025

AEI Vol.16 Issue 8

ISSN (Print): 2977-3911

ISSN (Online): 2977-3903

Download Cover

Abstract

In industrial manufacturing, accurate and efficient identification of product surface defects is essential for ensuring product quality, optimizing the production process and reducing cost. However, complex and diverse defect morphologies and the need for fine-grained description present significant challenges. General image description methods based on large visual language models often struggle to provide accurate defect type and location information for specific areas such as steel surface defect recognition. To address this, a defect identification method for the Qwen2.5-VL-3B large model based on LoRA fine-tuning is proposed. We built a specialized dataset covering six key steel surface defects—cracks, impurities, plaques, pitting, scale penetration, and scratches—and refined the model through efficient low-rank adaptation. Experimental results demonstrate that the fine-tuned Qwen2.5-VL-3B model significantly improves industrial defect recognition, accurately identifying defect types and locations, thus overcoming limitations of general large models and providing an efficient solution for industrial inspection.

Keywords:

steel surface defects, defect recognition, vision large model, LoRA fine tuning, defect location

View PDF

References

[1]. Saberironaghi, A., Ren, J., & El-Gindy, M. (2023). Defect Detection Methods for Industrial Products Using Deep Learning Techniques: A Review.Algorithms, 16(2), 95. DOI: 10.3390/a16020095

[2]. Chen, F., Fu, L., Zhang, Y., Li, J., Zhang, Q., & Bi, S. (2025). A Review of Deep Learning-Based Steel Surface Defect Detection.Academic Journal of Science and Technology,15(1), 198-202. DOI: 10.54097/g36nm962

[3]. Ghosh, A., Acharya, A., Saha, S., Jain, V., & Chadha, A. (2024). Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions. arXiv preprint arXiv: 2404.07214. arXiv: 2404.07214

[4]. Qwen Team, Alibaba Group. (2025). Qwen2.5-VL Technical Report. arXiv preprint arXiv: 2502.13923. arXiv: 2502.13923

[5]. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., ... & Chen, W. (2022). LoRA: Low-Rank Adaptation of Large Language Models. International Conference on Learning Representations (ICLR). arXiv: 2106.09685

[6]. Lei, S., Hua, Y., & Zhihao, S. (2025). Revisiting Fine-Tuning: A Survey of Parameter-Efficient Techniques for Large AI Models. Preprints.org. DOI: 10.20944/preprints202504.0743.v1

[7]. Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.Advances in Neural Information Processing Systems (NIPS), 28, 91-99.

[8]. Lester, B., Al-Rfou, R., & Constant, N. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning. arXiv preprint arXiv: 2104.08691. arXiv: 2104.08691

[9]. Wang, C. Y., Bochkovskiy, A., & Liao, H. Y. M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv: 2207.02696. arXiv: 2207.02696

[10]. Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.Advances in Neural Information Processing Systems (NIPS),28, 91-99.

References

[1]. Saberironaghi, A., Ren, J., & El-Gindy, M. (2023). Defect Detection Methods for Industrial Products Using Deep Learning Techniques: A Review.Algorithms, 16(2), 95. DOI: 10.3390/a16020095

[4]. Qwen Team, Alibaba Group. (2025). Qwen2.5-VL Technical Report. arXiv preprint arXiv: 2502.13923. arXiv: 2502.13923

[6]. Lei, S., Hua, Y., & Zhihao, S. (2025). Revisiting Fine-Tuning: A Survey of Parameter-Efficient Techniques for Large AI Models. Preprints.org. DOI: 10.20944/preprints202504.0743.v1

[8]. Lester, B., Al-Rfou, R., & Constant, N. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning. arXiv preprint arXiv: 2104.08691. arXiv: 2104.08691

Cite this article

Liu,J. (2025). LoRA fine-tuned Qwen2.5-VL large model for accurate description and location of steel surface defects. Advances in Engineering Innovation,16(8),100-107.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

About volume

Journal: Advances in Engineering Innovation

Volume number: Vol.16

Issue number: Issue 8

ISSN: 2977-3903(Print) / 2977-3911(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:

1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.

2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.

3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).