Development of portable automatic number plate recognition (ANPR) system on Raspberry Pi

ABSTRACT


INTRODUCTION
Automatic Number Plate Recognition (ANPR) technology has played a substantial role in automated traffic law enforcement. An ANPR system is essentially a means of identifying a vehicle number plate by extracting the information from an image file using image processing techniques. The process typically consists of image acquisition, image preprocessing, determination and extraction of region of interest (ROI), and interpreting the pixels into numerically readable characters using optical character recognition (OCR) [1]- [3].
There are many techniques used to improve the accuracy of the system; some common ones are RGB, YcbCr, image filter, fuzzy algorithm, image binarization, support vector machines, genetic algorithm and artificial neural network [4], [5]. This paper aims to develop an ANPR system running solely on the Raspberry Pi using OpenCV.
Raspberry Pi is a card-sized cheap mini-computer aimed to make computing accessible to the public. While the original intention of the Raspberry Pi is to provide a base for kids to learn programming, it is also gaining popularity among tech-enthusiasts as it can be used to do different types of commercial programming. It serves as an efficient base due to its low cost and the number of interfaces available. The Raspberry Pi can be used instead of a personal computer, but with some limitations due to its limited processing power [6], [7]. Table 1 further illustrates the specification details of the Pi 3 used in this project. An 8 megapixels (MP) Raspberry Pi camera can be attached to the on-board Raspberry Pi camera connector, and this creates an image capture system with embedded computing that can extract information from images without the need for an external processing unit. The multiple GPIOs available can interface with external devices and can be used to make results available to other devices. Considering the requirements of image processing compared to the Raspberry Pi's processing module and its peripherals, it is decided that the system is capable on executing the tasks specified. Experimental results show that the designed system is decent enough to run the image capturing and image recognition algorithm [8].

PROPOSED DESIGN
The overall system design can be categorized into 2 parts, the hardware design and the software design. The hardware design as depicted in Figure 1 will be entirely run on a Raspberry Pi 3 with Raspbian Jessie OS installed, with additional peripherals of an 8 MP camera to capture images, and a 3.5" TFT LCD to display the results. The recognized number plates will also be logged to a cloud database inside Pi. The system is intended to be portable so it is powered by connecting it to a mobile powerbank. In addition to the Python programming language that is natively used in Raspberry Pi environment, this system will also use OpenCV, a computer vision simulation from Intel [9] to help with image preprocessing that is designed with criterias of resource optimization, low power consumption and improved speed. The proposed portable ANPR system software implementation can be categorized into 7 procedural steps as illustrated in Figure 2. Each process is crucial because the result of one particular process will be delivered on to the next process.

Image acquisition
An 8 MP Pi NoIR camera capable of taking infrared photos up to 3280×2464 pixels is used. This camera is also capable of capturing video at 1080p30, 720p60 and 640×480p90 resolutions which is a high quality video [10]. To reduce the computational load on the Raspberry Pi, this camera is setup to capture images with only 640×480 pixels. Before testing the system with a live image, a test image is preloaded to test the algorithm as shown in Figure 3.

Image desaturation
The next step is converting the colour image into grayscale by applying image desaturation [11]. This is done to reduce the complexity of processing colour image. A grayscale digital image is a single sample, carrying only intensity information, and this can facilitate the processing better [12]. OpenCV has the cvCvtColor function to convert color images to grayscale and the effect of this function is demonstrated in Figure 4.

Image thresholding
Thresholding is the process of converting the grayscale image into a bi-level image. It does not identify main objects, but rather separate them from the background in order to obtain the information that we have to deal with. During thresholding, it is crucial to set the correct threshold value which will best determine a pixel as an object or a background. We use OpenCV's cvThreshold to achieve this, and the result is as illustrated in Figure 5.

Gaussian filter
Gaussian filter is blurring an image by applying a Gaussian function. It is used to reduce image noise and reduce unwanted detail. The visual effect of this blurring technique is a smooth blur resembling that of viewing the image through a translucent screen. Gaussian filter is typically used as a pre-processing stage in computer vision algorithms in order to enhance image structures at different scales [13]. The effect of applying this function is demonstrated in Figure 6.

Morphological transformation
Morphological transformation inspects geometrical structure within an image by probing it with small patterns called structuring elements. The result is a nonlinear image operator that is well-suited for exploring geometrical and topological structure. The operator is applied to an image in order to make certain features apparent, and distinguish meaningful information from irrelevant distortions by reducing it to a skeleton [14]. This process has 4 kinds of operations: expansion, corrosion, opening and closing operation. Figure 7 shows the effect of the image after morphological transformation.

Segmentation
Image segmentation is the process of dividing an image into multiple parts, typically used to identify objects or other relevant information in digital images [15]. After morphological transformation, 2 segments were identified as potential candidates as depicted in Figure 8. The segments will then go through another process where the system will determine which one has the greatest possible character. The segment will be isolated in a square image, and it is tested so that the longest list of potential character will be determined as the actual number plate. The last image with the most potential characters in it will then be chosen to go for character recognition in the next process.

Character recognition
To recognize alphanumeric characters from the segmented image, the potential number plate is segmented further into individual characters before OCR testing as shown in Figure 9. This system uses the K-Nearest Neighbour (KNN) method which requires running a set of test on the character library against the pixels of the separated individual characters from the image [16]. A similarity function () can be derived from a distance function (). When ( 1, 2) [0,1] we define the similarity as ( 1, 2) = 1 − ( 1, 2) ℎ 0 ≤ ( 1, 2) ≤ 1 1 2 The system will determine the output of the alphanumeric number plate by printing it on the image. KNN algorithm operates by giving a set of training data to generate the data structure. For this system, multiple data structure has been generated to get the highest percentage of accuracy of the character recognition [17].

EXPERIMENTAL RESULTS AND DISCUSSION
The system is then set to capture images from the calibrated [18] Pi NoIR camera and samples of 100 images were captured and tested. It is observed that the system manages to deliver good results when the subject is within 2 meters from the camera. Samples of the successful recognitions image were as illustrated in Figure 10 Figure 13. Successful recognition Sometimes the system fails to recognize characters correctly, where letters are filtered out causing the character not to be registered in the data structure as depicted in Figure 14. In some cases, the system confuses similar letters such as D from O or 6 from 8 and so on. This is illustrated in Figure 15 and ultimately in Table 2.
Based on 100 samples taken, it is found that the processing on the Raspberry Pi 3 took 2 to 3 seconds to process the image and come out with the recognized number plates, with a success recognition rate of 85%.

CONCLUSION
The development of a portable ANPR system using Raspberry Pi 3 is demonstrated in this article, implementing OpenCV as the core image processing software. The system performs magnificiently, but it is hoped that the Raspberry Pi foundation can come out with a newer model with better processing power to eliminate the 3 seconds delay needed to produce the results. In the meantime, future work may be done to improve the accuracy in the recognition process as well as speed up the time taken to produce the output on the algorithm side.