Automatic face and VLP’S recognition for smart parking system

One of the concerning issues regarding smart city is Smart Parking. In Smart Parking, some researchers try to provide solutions and breakthroughs on several research topics among security systems, the availability of single space, an IoT framework, etc. In this study, we proposed a security system on Smart Parking based on face recognition and VLP’s (Vehicle License Plates) identification. In this research, SSIM (Structural Similarity) method as part of IQA has been applied due to its reliability and simple computation for face detection and recognition process. From the test results of 30 data, obtained the highest SSIM value 0.83 with the highest accuracy rate of 76.67%. That level of accuracy still has not reached the implementation standard of 99.9%. So that it still needs to be improved in the future studies, especially in the filtering noise section


Introduction
IoT based smart parking system is a system to monitoring the occupancy that processed information from the sensors which deployed in the parking area [1]. Depart from previous research [2] the image processing method that used in face detection and face recognition was Image Quality Assessment. (IQA) Image quality assessment plays an important role in visual signal communication and processing such as biomedical imaging, printing and display systems, image fusion, segmentation, and image acquisition [3]. In general, PSNR, MSE, SSIM, and RMSE are commonly used to compare the quality between enhancing the image with the original image, while the advantages of SSIM over others are more consistent from the human subjective perception [4,5].
Some experiments show that SSIM was one of the successful methods for IQA [6]. Among them [7], SSIM has successfully approached the analysis of medical imaging especially in noised image such as Rician noise in MRIs and Poisson noise in X-rays, but still failed when implemented on some blurred medical images. Other experiment [8] by Varnan, showed that SSIM was better than MSE and PSNR when testing the accuracy of a distortion image. Based on some experiment results above, thus SSIM became the mainstay method in this research which has been used for face recognition process in smart parking system.
Another part of smart parking system is vehicle license plates recognition (VLPs), many researchers studied how to get high accuracy of plate recognition [9] by simulating several methods such as MIP (Morphological Image Processing, 96.4%) [10], Blob extraction and segmentation method (90.0%) [11], OpenCV [12][13][14], and some enhancement methods like OCR (Optical Character Recognition) [15,16]. While the successful of the VLPs recognition problem as part of Machine Vision System (MVS) will depend on how much about the problems domain can be solved [17]. Based on the explanation above in this study, we proposed a face and VLP's recognition system using OpenCV based on SSIM approached. OpenCV method was used because often used in real time conditions. In this study, the test limits only one car in one queue of vehicles and only one face in captured image.

Research Method 2.1. System Design
The system design consists of entrance gate, main controller unit that connected to the server and data base, and the end part is gate. At the entrance gate, camera 1 will capture the plate image, then saved according to the captured time. Camera 2 performs the capture process also on the driver's face to be stored in the entry face dataset. After that, both data directories are stored in database and the entry primary gate will open as seen in Figure 1. When the vehicle will go out, the vehicle stops at a spot in front of the exit parking gate. The camera will capture the license plate and driver's face. The next process is face detection by matching face images whne captured on the exit gate with face images when entering the parking lot. This process is calculation of SSIM and MSE of both images, if the SSIM value is more than threshold then the face that recognice will be considered as a valid data and the gate will open and if the SSIM value is below the threshold the gate will not open. The license plate identification process is also done by matching the plate number when the vehicle enters the parking lot. This process is done by using OpenALPR (Automatic license plate recognition) application, the characters on the number plate image will be detected and then matched with the number plate that has been stored on the data base before. In this research, for data base design consist of seven field id they are; id, time_in, time_out, pict_in, pict_out, face_in, face_out, no_plate where each filed has a data type: time_in and time_out using timestamp, no_plate, face_in, face_out using varchar, pict_in and pict_out using text.

Vehicle and Face Detection
The algorithm that used to detect the existence of a license and face object was the Haar Cascade algorithm. The Haar method was used for segmentation of object (vehicle and face) on image due to its superiority which can segment the characters on license plate in real time condition as tested by Chirag [18] in his survey research for Automatic Number Plate Recognition (ANPR). Object classification was using a fixed scale parable, in this study using 360x240 pixels. If the car object in the image was bigger or smaller than the pixel size then the classifier will continuously run several times to search the car object on the image during the detection process. Once the vehicle object is detected, the next step is face detection. In the process of haar cascade for vehicle detection requires a positive image and negative image, then openCV will generate as sample using opencv_createsamples for both of images which containing a binary format. Positive image is the image that there are objects to be searched while the negative image is a background. Next, train the object by running opencv_traincascade, this library serves to generate files with extension .xml that can detect the existence of the vehicle when the camera takes pictures or video.

Face Recognition
The used method in face recognition process was SSIM (The Structural Similarity Index). The SSIM algorithm works by comparing the luminance (l), contrast (c), and structural (s) of a host image (x) with distorted or enhanced image (y) [19][20][21]. In general, the equation of SSIM is shown as follows [22]:   Figure 2, the luminance changes are influenced by the mean intensity while the contrast and structural changes are influenced by the value of covariance. Figure 3 below illustrates how SSIM values are obtained. To get its SSIM value, the first step was to measure the scale of the lighting value luminance (x, y), then calculated the contrast value from both contrast (x, y) images. While to get the value of structure comparison c (x, y) the lighting value divided by contrast value. All three values were combined so that the SSIM values appear from both images.

License Plate's Characters Identification
In the ALPR process, the system detects the number plate characters already known by the system. To simplify the detection of plate characters, in OpenALPR the OCR library is added to determine the dimensional and character configurations of the vehicle license plate. In this proposed system, the first process is to detect a vehicle object using the Haar Cascade algorithm. This algorithm works by analyzing each image location and classifying it as a car or 1701 not a car. Classification of objects using a fixed scale and generate files cars .xml, in this research using the image size 360x240 pixels. The second process, OpenALPR analyzes the image to identifying the license plate. The result is a writing that represents many of the number plate characters. In the process of detecting the number plate characters needed is the train data file of the Indonesian license plate character, the config file and which is in the runtime_data directory. The Indonesian number plate pattern in OpenALPR is defined as "id @####@@" or "id @####@@@ ", where @ as letter and # as number that declares variable results in output program.

Test Parameters 2.5.1. Character Detection Accuracy of License Plate
To determine the accuracy of license plate, the system will read a video input of a vehicle recording. Number of input objects read as many as 30 samples from the 30 samples will be calculated how many the proposed system read the plate correctly and how many the license plate is detected not in accordance with the actual conditions. The accuracy calculation as shown in (2).

Accuracy of Face Recognition
To determine the accuracy of face recognition. The program will read thirty different drivers' faces according to the condition of the driver's exit from the parking area. After that it will get the average value for each SSIM of the same 30 drivers. (3) shown the average value for SSIM we call mean SSIM.

Result and Analysis
Implementation of face and plate detection system in this research was at a car park in Telkom University field. A digital camera mounted on a pole with a height between 1-1.5 meters was adjusted to the height of the car. The camera placement can be seen in Figure 4 and Figure 5. The application test is performed using video samples recorded through the camera and in real time using a webcam. When the program is run it will display the video being called, in the video there is a vehicle object that will be detected by OpenCV. The sample detection plate process can be seen in Figure 6. Furthermore, the process that occurs is the detection of characters on the vehicle license plate run by OpenALPR. Once the plate is detected and characters are identified, then the data will be stored in the database to match when the vehicle is go out. The Application  Table 1, true positive value was 23/30 or 76.67% out of 30 test objects and false negative 23.33%. So, it can be concluded that system sensitivity of license plate identification was 76.67%. Success rate is strongly influenced by environmental conditions ie the intensity of light. When the intensity of the light is too high can cause light to bounce on the plate so that the character of the plate will be difficult to recognize. As for the analysis of the introduction of each character, it is difficult to distinguish between "1" and "I", "8" and "B", and also "0" and "O" because they have similar feature.  Based on the testing, the fastest time can be reach at 64 seconds, while the longest time of 91 seconds while for the average time is 78.76 seconds. System performance test of face recognition was performed to know the accuracy level with matching the face of the car driver. This test was done through three stages as follows: a) First Stage (system process on entrance gate) Face image obtained after the camera takes a picture of the driver then run the algorithm to perform the process of auto cropping. In this pre-processing, the system also zoomed parts of the face to simplify the recognition process as shown on Figure 7.
After the pre-processing, then the face image will be stored in the dataset in jpeg format and with the file name that has been determined will be compared to the face image when the driver get out from the parking lot. b) Second Stage (system process on exit gate) At this stage, when the car getting out at the parking lot, the system will capture the driver's face on the area surrounding the gate. The captured image of the driver then become input on the pre-processing and image matching process on the face image at the At this stage, an accuracy test was conducted by calculating the SSIM value on face image when entering and getting out the parking lot. This test was conducted to get the SSIM value of a number of faces that use parking facilities that used for database. A total of 30 car drivers were tested on this experiment. The system tested by placing the camera at 12:00:00 PM and the set distance was ±50 cm. Table 1 shows the test results on SSIM values as well as gate conditions. Figure 8 shown that SSIM value between face entrance and face out was 0.67, then the gate will get status "open".  Table 2 and Figure 9, the average value of SSIM is 0.58 where a total of 23 driver's faces were identified well at the exit gate. SSIM values below 0.5 on the other 7 faces caused failure to access out of the gate. This low SSIM value due to improper object position when the taking face was done. From this test obtained 76.67% of accuracy. The rate of accuracy system can be improved by testing several parameters like background subtraction, decrease level of contrast with median filtering, up and down the position of webcam more precision.

Conclusion
This research has successfully implemented face and vehicle license plates (VLPs) identification for smart parking system at the parking lot in Telkom University area. This research applied SSIM algorithm as the main core in face recognition process that achieve accuracy rate of 76.67% on 30 samples of vehicles with the highest SSIM value obtained was 0.83. The best SSIM value is achieved when taking the driver's face in the right position with the same contrast and lighting. The failed recognitions on some of the driver's faces probably due to improper object position when the camera takes a face. This research is still being developed with a focus on the process of image acquisition and the effect of contrast and noise filtering.