1. Introduction
Congestion in major cities has been a growing concern for some years now. Automobiles have surpassed other modes of transportation in popularity. Building "smart" cities is necessary to improve urban transportation automation and management. As science and technology advance, licence plate readers play an increasingly important role in modern cities. The primary applications of licence plate recognition are automated vehicle identification and monitoring based on licence plate data. Analyse vehicle images or video sequences captured by surveillance cameras to get a unique number plate number and determine the position of the number plate to finish the recognition process. In the future of intelligent transportation and traffic data management, licence plate recognition will play a crucial role.
In broad terms, a comprehensive licence plate recognition system comprises five components, namely, the acquisition of primary licence plate images, image preprocessing, licence plate localization, character segmentation, and character identification. Based on variations in design concepts across different techniques for licence plate positioning and character recognition, it can be inferred that the current advancements in licence plate recognition and positioning are as follows: (1) Licence plate positioning Early licence plate positioning methods mainly used licence plate positioning to find the licence plate area and extract its background colour to obtain a binary image where each pixel is black or white. On the basis of binary images, character height, width, and tilt angle are detected to obtain accurate character height and tilt angle of the licence plate area and width estimation values of the licence plate character area. The current licence plate location methods are mainly classified into edge-based detection of licence plates, colour-based detection of licence plates, texture-based licence plate detection, and character-based licence plate detection. (2) Licence plate recognition The early character recognition method was based on character normalisation and template matching [1], which aims to obtain a template that matches the licence plate characters, further determine the recognised characters for grammatical analysis, and obtain more reasonable recognition results. The template matching approach used in the character recognition methodology only requires extracting the global features of the character region without performing character segmentation. For high-quality images that conform to the template criteria, the precision of licence plate recognition technology is high and fast. The drawback is that if the licence plate image has low quality or resolution, the recognition rate will significantly drop or even fail to recognise it. The selection and production of standard templates also require repeated experiments to ensure stable performance. According to different character recognition methods, today's character recognition methods are primarily classified based on their use of template matching techniques and feature statistics for character recognition, and the present study focuses on character recognition utilising convolutional neural networks such as target detection algorithms YOLO and SSD algorithms [2],[3].
Focusing on the above two aspects, this paper provides a detailed overview of vehicle identification in Section 2, which includes the basic processes, classic frameworks for licence plate positioning and character recognition, representative methods, and their advantages and disadvantages. In Section 3, a quantitative comparison of existing representative methods is conducted. In Section 4, we discuss the challenges and future developments in the research field of vehicle licence plate recognition. Finally, we summarise the main points of this paper in Section 5.
2. Method
This section provides a comprehensive overview of the recent advancements in licence plate recognition, encompassing image preprocessing, licence plate localization, character segmentation, and character recognition. The discussion follows the fundamental pipeline of licence plate recognition.
2.1. Image preprocessing
Licence plate recognition first requires obtaining an image of the licence plate. We need to preprocess the original licence plate image to reduce interference factors before licence plate positioning and ensure image quality. Preprocessing methods include preprocessing colours to grayscale the image, binarizing the image through colour information, correcting licence plate tilt, and so on. A representative paper on image preprocessing for the process of identifying and reading licence plates using optical character recognition technology is "Research on Pre-Processing Methods for Licence Plate Recognition" [4]. The present investigation explores various aspects pertaining to licence plate recognition systems, including image preprocessing, character segmentation, and licence plate localization. Convolutional neural networks (CNN) are utilised in the image preprocessing stage to implement edge detection techniques. The process of character segmentation is accomplished by utilising character vertical projection technology in conjunction with character boundary recognition. Mathematical morphology and statistical jump points are both used in the process of locating licence plates, which requires the employment of certain procedures. This method has shown its effectiveness in practical scenarios.
2.2. License plate location
2.2.1. Algorithms for locating licence plates based on edge detection. License plate location algorithms based on edge detection typically involve several stages, including image preprocessing, edge detection, and license plate location. Initially, the image is converted to black and white and then binarized. Next, an edge detection operator like the Sobel operator utilised to identify the boundaries of the licence plate in the image.
An algorithm that employs an enhanced Roberts operator for edge detection and mathematical morphology to produce candidate areas of the licence plate is one approach for finding a licence plate based on edge detection [5]. This algorithm is one way for locating a licence plate. After that, the area corresponding to the licence plate is precisely found using statistical jump points. This algorithm has been shown to effectively suppress noise in images and shorten the time required for locating a license plate. The advantage of edge detection-based license plate location algorithms is that they can effectively suppress noise. However, these methods may be sensitive to changes in lighting conditions and may require careful parameter tuning to achieve optimal performance.
2.2.2. Algorithms for locating licence plates using texture and colour features. The location of the licence plate can be ascertained through a comparative analysis of the licence plate colour with that of the vehicle body. This requires high-quality images taken under good lighting conditions. Mathematical morphology operations are used to generate several candidate regions, which are then used to locate the licence plate based on horizontal edge segments and projection histograms. The pixel intensity in these candidate regions is then used to recognise the licence plate.
A commonly used approach for detecting licence plates is through colour-based location methods, such as the algorithm proposed in [6]. This algorithm has two steps: detecting vehicles by subtracting the background; and the proposed methodology involves the integration of texture and colour characteristics for the purpose of identifying the licence plate region. It has been shown to be effective in overcoming limitations posed by complex backgrounds and achieving high detection rates in challenging situations. By considering both colour and texture distribution characteristics of the licence plate region within an image, algorithms that rely on colour and texture can effectively overcome limitations posed by complex backgrounds and achieve high detection rates in challenging situations. However, these methods may have some limitations as they require high-quality images of the licence plates, which may not always be available. The degree to which the original images of the licence plate are important for the effectiveness of these methods is important, which may limit their applicability in some scenarios.
2.2.3. License plate location based on characters. The licence plate image is segmented into characters using methods such as template matching or feature matching. The licence plate location is then identified using methods such as artificial neural networks.
A representative method for licence plate location based on characters is the algorithm proposed in [7]. The methodology being suggested utilises a cyclic generative adversarial network (GAN) to generate synthetic licence plate images that closely resemble authentic ones. This technique aims to address the issue of data imbalance and enhance the training dataset. The present study proposes a method for locating licence plates that incorporates an enhanced VGG architecture and the U-Net model [8],[9]. This method has been shown to effectively handle the issue at hand, which pertains to the inadequate identification of licence plates that are both inclined and of a smaller size. Character-based licence plate location algorithms have the advantage of being able to effectively handle images with complex backgrounds and poor illumination. However, the quality of the original images of the licence plate is important for the effectiveness of these methods, which may limit their applicability in some scenarios. Furthermore, if the segmented characters are incomplete or occluded, it may impact the precision with which characters are recognized.
2.3. Character recognition
2.3.1. Recognition of characters by a process of template matching. One of the first techniques that was used for character recognition involved template matching, which requires a standard library of samples that are matched with an input sample using image processing techniques. The provided input sample is cross-referenced against all template characters contained within the library [10]. The template character that exhibits the greatest degree of similarity is subsequently identified and designated as the outcome of the classification process. In their study, the authors suggested using an online character recognition system. The system employs templates that are generated through an automated process and can represent different writing styles for a specific character. The method computes similarities between an input character and a set of templates from each character class and identifies the character with the highest similarity measure. This method is straightforward because it uses binary images that reduce search space, resulting in relatively fast recognition speeds. However, it depends on library size and is sensitive to lighting conditions that can cause problems such as blurring and sticking. Finding a model with a high recognition rate and accuracy under low light intensity is challenging, requiring optimisation and processing to make this method more practical in future applications.
2.3.2. Identifying characters through the analysis of feature statistics. In the realm of character recognition, feature statistics serve as the basis for the identification process. The initial step involves the correlation of primary data points and the quantification of their characteristics in the character image to be recognised. Then, based on knowledge representation methods and statistical learning or algebraic feature representation methods, these character image features are classified and judged. The corresponding character features of the character image to be recognised are counted and placed in the character library for matching with the feature set to obtain the final matching result. It is mainly based on statistical and mathematical theory.
Qi et al. proposed a recognition method for feature extraction and selection for stroke-based text recognition [11], which is a machine learning-based approach that utilises various features to identify text in English generated by analysing the structural and statistical characteristics of strokes. The efficacy of the proposed methodology is assessed using a corpus of English textual images. The findings indicate that the method put forth attains superior accuracy in recognition when compared to alternative methods. This method is generally more advanced and complete at present. It can handle noisy characters well. However, in practical applications, there are some challenges due to various objective factors, such as the accumulation of multi-dimensional features and the insensitivity to small deformations. Therefore, there are often problems such as character breakage and blurring in the recognition process that affect the final results.
2.3.3. Character recognition based on Convolutional Neural Networks. The present neural networks utilised for licence plate character recognition predominantly comprise convolutional neural networks, neural networks that use feedforward data, recurrent neural networks, and fully connected neural networks. First, the neural network learns from the input sample data to collect the characteristics of the various character samples. The preprocessed features of the characters to be recognised are fed into the neural network for the purpose of training. Subsequently, the neural network employs a feature-matching process to recognise characters by comparing their distinctive attributes with those of the sample characters. Yang et al. apply convolutional CNN to recognise handwritten English characters [12]. In the literature, the paper first performs the preprocessing of the NIST dataset and then continuously optimises the CNN model used in the experiment [13]. At present, the application of convolutional neural networks is the most extensive, with good self-classification and learning abilities, and the running speed is very fast.
3. Performance comparison and analysis
The techniques delineated in the aforementioned article serve as the basis for the present analysis. We extracted and compared the experimental results of the methods, which can be seen in Table 1.
Table 1. The present study evaluates the efficacy of various representative methods.
Method | Categories of LPs | Number | Success Number | Success rate |
[5] | None | 120 | 109 | 90.833% |
[6] | white-blue | 300 | 295 | 98.33% |
back-yellow | 200 | 196 | 98.00% | |
[7] | None | None | None | 94.15% |
4. Discussion
In the current actual driving situation, the licence plate is mostly in a state of motion, tilted rather than directly observable. If it is closer, the camera angle deviation is large, and if it is farther away, there will be a blurry situation. In response to this issue, more and more researchers are starting to study how to train recognition models based on a limited quantity of licence plates. recognition annotation samples, that is, small sample licence plate image recognition [14]. The task of small-sample image recognition requires machine learning models that can be trained and learn effectively even when provided with a limited amount of labelled data. The N-way K shot form, in which N kinds of data are involved, is the most researched variant of this issue at the moment, and each type of data only contains K-labelled samples [15].
The existing small-sample image recognition problem can be seen as based on deep learning [16]. For the image recognition problem of degree transfer learning, we refer to the small amount of annotation data mentioned above as the target data domain, and the subsequent recognition tasks are all based on the target. Based on the categories contained in the target data, an auxiliary dataset that is mutually exclusive to the target data domain category is usually introduced. On the contrary, a small amount of annotation in the target data domain results in richer annotation samples and more categories in the auxiliary datasets. To solve the task of small sample image recognition in the N-way K-shot form [17], most methods learn prior knowledge from auxiliary datasets and then annotate with limited Utilise this prior knowledge to complete learning and prediction tasks in the target data domain.
The study of tiny sample sizes has seen significant growth in recent years and received much attention from researchers. Based on the summary of the small sample learning field, several development directions are proposed: (1) Neural network interpretability Despite the significant achievements of deep learning models in various fields, neural networks inherently possess a certain degree of opacity [18]. Further exploration of the interpretability of neural networks can enable researchers to make more informed structural or training method improvements based on the underlying mechanisms of deep learning for problems with limited samples. (2) More general, small-sample learning methods Although researchers are increasingly focusing on small sample learning problems across a range of tasks, these efforts are typically confined to specific task modes. However, a practical small sample learning system should be capable of addressing small sample recognition problems for arbitrary category label data. Furthermore, the small-sample learning tasks currently employed in research are essentially data partitions derived from complete large datasets and exhibit significant inter-task correlations. Conducting research based on more realistic small sample tasks and more loosely organised data is a critical step in advancing small sample research from theory to practise. (3) Incremental learning problems A small sample recognition system may initially face data scarcity [19], but as more data enters the system, the accumulation of annotated data within the system will increase. Effectively utilising this newly entered data to enhance and improve the current recognition system is crucial for the sustainability of small-sample learning systems.
5. Conclusion
At present, licence plate recognition technology is extensively employed in a variety of contexts, including parking management, weighing systems, static traffic vehicle management, highway overload control, highway inspection, vehicle dispatching, and vehicle detection. This technology plays a vital role in promoting automated traffic management, reducing traffic congestion, protecting the public in metropolitan areas, and enhancing traffic safety, highway overload control, highway inspection, vehicle dispatching, and vehicle detection. This technology plays a vital role in promoting automated traffic management, reducing traffic congestion, protecting the public in metropolitan areas, and enhancing traffic safety. Smart licence plate recognition cameras are continually evolving to meet changing market demands. In the future, an increasing number of specialised scenarios will require the use of dedicated smart licence plate recognition all-in-one machines. Such technology will enable licence plate recognition to contribute to the industrial revolution by providing new development directions and assisting enterprises in achieving big data management and industry reform and progress.