Large Language Models and Their Evolution

Zihan Zhou

doi:10.54254/2755-2721/2025.25586

Applied and Computational EngineeringOpen access

Large Language Models and Their Evolution

Research Article

Open Access

Large Language Models and Their Evolution

Zihan Zhou ^1*

¹ School of Computer Science and Engineering, Southeast University, Nanjing, 211189, Jiangsu, China

^*Corresponding author: Academic_paper666@163.com

Published on 30 July 2025

ACE Vol.173

ISSN (Print): 2755-273X

ISSN (Online): 2755-2721

ISBN (Print): 978-1-80590-231-7

ISBN (Online): 978-1-80590-232-4

Download Cover

Abstract

With the rapid advancement of artificial intelligence (AI), large language models (LLMs) have become the foundational infrastructure for natural language processing (NLP) research and industrial applications. By leveraging massive parameters and vast pre-training data, LLMs have significantly enhanced text understanding, generation, and cross-modal reasoning capabilities. This paper systematically reviews the technical evolution of LLMs from n-gram statistical models to the Transformer architecture, based on five key review papers. It analyzes training and alignment paradigms such as “pre-training & fine-tuning” and “RLHF/RLAIF”, as well as the exponential parameter expansion driven by the Scaling Law. Furthermore, we summarize the latest application progress of LLMs in code generation, intelligent customer service, medical and legal assistance, and other fields, and analyze the challenges they face in terms of data privacy, model bias, hallucination phenomena, and energy consumption. Finally, this paper proposes four research priorities for the future: first, leveraging explainable mechanisms to enhance model transparency; second, strengthening value alignment and security controls; third, exploring green and efficient model compression and inference schemes; and fourth, leveraging interdisciplinary collaboration to build the next generation of general-purpose intelligent systems that are both fair and sustainable.

Keywords:

Large Language Models, Transformer, Scaling Law, Pre-training & Fine-tuning, Prompt Engineering

View PDF

References

[1]. Jana S, Biswas R, Pal K, et al. The evolution and impact of large language model systems: A comprehensive analysis [J].Alochana Journal 2024.

[2]. Wang Z, Chu Z, Doan T V, et al. History, development, and principles of large language models: an introductory survey [J]. AI and Ethics, 2024: 1-17.

[3]. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need [J]. Advances in neural information processing systems, 2017, 30.

[4]. Beltagy I, Peters M E, Cohan A. Longformer: The long-document transformer [J]. arXiv preprint arXiv: 2004.05150, 2020.

[5]. Naik D, Naik I, Naik N. Large data begets large data: studying large language models (LLMs) and its history, types, working, benefits and limitations [C] The International Conference on Computing, Communication, Cybersecurity & AI. Cham: Springer Nature Switzerland, 2024: 293-314.

[6]. Kaplan J, McCandlish S, Henighan T, et al. Scaling laws for neural language models [J]. arXiv preprint arXiv: 2001.08361, 2020.

[7]. Liu Y, Han T, Ma S, et al. Summary of chatgpt-related research and perspective towards the future of large language models [J]. Meta-radiology, 2023, 1(2): 100017.

[8]. Wei J, Bosma M, Zhao V Y, et al. Finetuned language models are zero-shot learners [J]. arXiv preprint arXiv: 2109.01652, 2021.

[9]. Jang J, Ye S, Yang S, et al. Towards continual knowledge learning of language models [J]. arXiv preprint arXiv: 2110.03215, 2021.

[10]. Jacovi A, Goldberg Y. Aligning faithful interpretations with their social attribution [J]. Transactions of the Association for Computational Linguistics, 2021, 9: 294-310.

[11]. Zheng Z, Ning K, Wang Y, et al. A survey of large language models for code: Evolution, benchmarking, and future trends [J]. arXiv preprint arXiv: 2311.10372, 2023.

[12]. Taylor R, Kardas M, Cucurull G, et al. Galactica: A large language model for science [J]. arXiv preprint arXiv: 2211.09085, 2022.

[13]. Aoki G. Large Language Models in Politics and Democracy: A Comprehensive Survey [J]. arXiv preprint arXiv: 2412.04498, 2024.