Cyber-physical system based on image recognition to improve traffic flow: A case study

Vehicular traffic in metropolitan areas turns congested along either paths or periods. As a case study, we have considered a mass transport system with a bus fleet that rides over exclusive lanes across streets and avenues in an urban area that does not allow the circulation of lightweight vehicles, cargo, and motorcycles. This traffic flow becomes congested due to the absence of restriction policies based on criteria. Moreover, the exclusive lanes are at ground level, decreasing lanes for other vehicles. The main objective of this proposal consists of controlling the access to the exclusive lanes by a cyber-physical system following authorization conditions, verifying the permission status of a vehicle by the accurate recognition of license plates to reduce traffic congestion. Therefore, in the case of invading an exclusive lane without permission, the vehicle owner gets a notification of the fine with the respective evidence.

INTRODUCTION Metropolitan areas usually have high occupancy vehicle lanes duly identified based on criteria or conditions, e.g., when the driver of a lightweight vehicle travels with at least one passenger. In the city of Guayaquil, Ecuador, the mass transport service, named Metrovia System, is a solution for the mobility of ground transportation passengers, either from a bus stop (taking a feeder bus to arrive at an integration station or secondary station) or directly from an articulated bus (from a secondary station for moving to an specific area). An articulated bus transits along an avenue or street lane, occupying an exclusive ground lane. The density of an articulated bus is nearly 160 passengers in maximum capacity, which is higher compared to the number of people traveling in non-exclusive lanes. These lanes are shared along some paths with other vehicles with the authorization of the transit agency [1]. The local municipality dictates ordinances for the use of the exclusive lanes, applying a fine to unauthorized vehicles that enter, invade or obstruct them as shown in Figure 1, which consists of one vital minimum wage [2][3][4] (currently equivalent to USD 400), and two vital minimum wages in case of recidivism [4]. Photo capture from a closed-circuit surveillance system evidences the infraction, notifying to the offenders through electronic means if possible, or communicating them at the time they approach the transit agency for any process.
An argument for setting an exclusive lane is the efficiency in terms of traveling time of a higher density of users. At the same time, even if the route frequencies are well scheduled, the buses usually experience delays. Controversies highlight because of derived problems in non-exclusive lanes such as traveling time inefficiency due to traffic congestion, which may increase along some trunks at rush hour; more fuel consumption and Ì ISSN: 2088-8708 mechanical parts wear due to continuous jams; noise contamination and gas pollution. Because of traffic jam experiences, lane exclusiveness takes out during some periods and paths. As traffic intensifies, traffic officers allow lightweight vehicles to transit exclusive lanes [5] as traffic intensifies, whether reducing or intensifying congestion, not providing effectiveness and efficiency. Researches pursue traffic allocation to less congested routes, optimizing schemes for rerouting traffic, socializing direct benefits to drivers for enhancing path choices. Vehicular congestion issues can be addressed by system automation to provide alternative paths through not congested roads [6]. This work is organized as follows: Section 2 presents related work aspects, addressing that our case study focuses on occupancy controlling. Section 3 depicts the system design covering the image management process by databases for detecting occupancy violations and for the notification process to a vehicle's owner. Section 4 describes the analysis of the results considering false positive situations in captured images. In the end, Section 5 presents conclusions and lessons learned throughout the development of this proposal.

2.
RELATED WORK A related work directly associated with the Ecuadorian situation for controlling the lane access of a mass transport system, particularly in the city of Guayaquil, has not been developed. Nevertheless, similar studies in other countries are available, approaching security by controlled systems for persuading drastic lane changes, such as collision avoidance. These studies look forward to a confidence provision to decrease stress levels of drivers, helping them to detect situations for collision prevention, such as the researches presented in [7] and [8]. On the other hand, we present a cyber-physical system that allows lane sharing to optimize traffic flow circulation, considering lanes that are on a ground-level not occupied all the time.

SYSTEM DESIGN
The system design implements an electronic circuit with infrared (IR) diodes for vehicle detection, a Raspberry Pi camera [9] for photo capturing, and a Raspberry Pi 3 board for a validation process of the occupancy of the exclusive lanes as shown in Figure 2. We use imported libraries that work with MySQL databases to send text messages through a messaging platform. Both the photo capturing and the license plate validation allow the image reading. Three-dimensional arrays are set up for representing colors where rectangles are formed to indicate the position of the license plate and the color font of the signboards that overwrite the generated images. Moreover, the system reads a signal from an electronic circuit with IR diodes, which act as an infrared barrier to detect vehicles that ingress to an exclusive lane for taking a license plate photo. It is necessary to connect a pin of the Raspberry Pi 3 for the corresponding signal conveying through the IR diode [10,11] by the General-Purpose-Input-Output pin (GPIO 23) variable definition to count the fines and set anti-rebound in the electronic circuit. The Raspberry Pi camera works as an array for photo capture using searching functions. The array images successions depict real-time video recorded by the capture function that takes the photo at the instant the barrier determines it [12,13]. To get an appropriated image is necessary to adjust some camera features such as the resolution (320, 240), the angle of view (in this case 180 • ) and the frame rate to the maximum, aiming to capture the photos as fast as possible, setting it the maximum of 15 fps. The programming of an infinite loop allows the photo-taking, continuously displaying them into a frame. The setup of an anti-bounce code determines if a vehicle passes through the barrier for the photo taking. The process carries out for once until the vehicle passes the barrier as shown in Figure 3, saving the information for the license plate recognition.  An algorithm processes the photos based on neural networks trained for character recognition [14]. Data acquisition and an OpenCV library training function return a Boolean value: If false, there are problems in the file loading process; otherwise, if true, the process continues asking the photo captured after passed the infrared barrier, saving it into the coding files to identify the license plate, creating copies to generate the fine. For other sensed vehicle, the photo overwrites the photo captured at that instant. A reading function of the OpenCV library loads the photo image [15], and a detection function returns a region list where a rectangular area for license plate recognition finds alphanumeric characters whereas a preprocessing function performs color conversion to HSV in the photo image by extracting values [16] from the OpenCV library [15]. A segmentation function links to the photo image channels, setting addition and subtraction operations to maximize contrasts for removing gray-scale image noises [17], converted previously into erosion and dilation. Moreover, the first filtering removes the Gaussian noise, making it possible to change images to gray-scale as shown in Figure 4(a) for comparing each pixel of the threshold and getting its binary form as shown in Figure  4(b). With the binary image, the algorithm starts searching the license plate varying the gray-scale image and marking rectangles until finding the largest one as depicted in Figure 5.
Once detected and extracted a license plate image, the system proceeds to turn it to a gray-scale as shown in Figure 6(a). Character recognition starts with a license plate recognition function as shown in Figure  6(b). Previously, the creation of a back-up is necessary due to the binary image can alter the contours. Then, the analysis of valid characters by a neural network learning provides the files loaded when the process starts. If it is into the delimited area of the license plate, the recognition analyzes a returning character list. The area that contains the non-null characters in the license plate region draws a rectangle around it. Both console and dated images present the identified license plate as shown in Figure 7, displaying into an original image window and saving it into a folder if the process analysis determines the permission status has expired. A database that stores the information links a MySQL connector with the hostname to validate a username and a password, specifying the database name that saves the table with the vehicle information that passed with permission. A cursor creation is necessary to specify the lecture parameters for information depiction by console regarding a vehicle that has occupied the exclusive lane. The action selects license plate information depicting as a string per console as shown in Figure 8, and checking permission parameters and its use. Additionally, the system verifies whether the permission is expired or not by comparing its validation period with the current date. Registry commands use the updated information for depicting information through the database (e.g., name, e-mail, cell phone number, permission validity) of the authorized vehicles that paid the fee to occupy the exclusive lanes as depicted in Figure 9. An object calls a method to use sending arguments such as subject and recipient for the e-mail notification process, previously associating an account with the mail server for linking the object argument for the e-mail recipient.  For loading captured images with the modifications made in the analysis process, another object indicates the path where the image loads, adding a file name to the header and continuing with an authentication process by the configuration of an SMTP client session object that sends e-mails. Furthermore, a command with parameter information of the sender (administrator) and the recipient (database) sends an e-mail retrieving the image with relevant information of the vehicle that invaded the exclusive lane as shown in Figure 10, closing the previous object created.

Ì
ISSN: 2088-8708 Figure 10. E-mail with information of the fine Short message service (SMS) notifications ask some parameters registered in the database, such as information of a cloud account with the respective authorization [18]. A client parameter function object sends a message specifying the recipient and the cell phone number registered in the system warning to the owner that his/her vehicle has occupied an exclusive lane with an expired permission as shown in Figure 11. After sending a notification, another table adds information about the number of the generated fines such as the date and time of the infraction, image name, user information, and the license plate as depicted in Figure 12. Case studies where SMS notifications were deployed in different scenarios with the respective results are available in [19][20][21][22][23][24].

ANALYSIS OF RESULTS
Tests were carried out in outdoor environments during daytime hours to ease the reading of the license plates, with a scenario that resembles external conditions. The results of the captured images had variations due to either the position or the type of character in the plate. A different camera than the one used in the prototype system to check the coding efficiency captured the images of the license plates. At the moment of reading an image after loading previous ones, the program recognized characters that did not exist, presenting a false positive as shown in Figure 13. For example, depending on the font used in the license plate, letters with closed-form like the letter C and G depicted a distortion during the recognition as shown in Figure 14, delivering wrong characters such as 0 and 8, respectively. Alteration during the recognition process occurs depending on the inclination angle of the captured image [25,26], limiting the recognition area or altering the characters. There are successful cases where an image is captured with either a front or a rear profile without inclinations and with standard font letters with adequate accuracies, such as the ones mentioned in [27]. In our case study, a factor that affects the accuracy relationship was the inclination angle during the capturing of the characters as shown in the Figures 15(a) and 15(b). Table 1 depicts a match percentage of the algorithm when recognizing the characters from all the captures done.

LESSONS LEARNED AND CONCLUSIONS
The prototype was built using Python and Raspberry Pi 3 with neural network algorithms. According to the tests carried out, we obtained an accuracy rate average of 78.04%, a percentage not reliable for this sort of application. Future work should include algorithm testing that not only work with images in the front and rear profile but also to correct inclinations for a better image cleaning before the characters' extraction. On the other hand, the implementation costs can diminish by either doing the detection by Python codes or decreasing the capturing time of the license plate. Moreover, the Raspberry Pi camera was a convenient tool for capturing images due to its resolution of 5 Megapixels and static image processing of up to 2592x1944 pixels. However, other options can be considered for capturing motion images, to attain instant captures with higher resolution and expanded coverage to overcome speed constraints.
Regarding the text message notification platform, its scope was limited insomuch as it was not able to work worldwide, covering to cell phones of the mobile network operators working in Ecuador. This situation can be overcome with the implementation of GSM modules. Also, as this project works through the Internet, it is required a system connection with enough bandwidth to ensure notification delivery in case of generation of a fine.