Gazing Time Analysis for Drowsiness Assessment Using Eye Gaze Tracker

From several investigations, it has been shown that most of the traffic accidents were due to drowsy driving. In order to address this issue, many related works have been conducted. One study was able to capture the driver’s facial expression and estimate their drowsiness. Instead of measuring the driver’s physiological condition, the results of such measurements were also used to predict their drowsiness level in this study. We investigated the relationship between the drowsiness and physiological condition by employing an eye gaze signal utilizing an eye gaze tracker and the Japanese version of the Karolinska sleepiness scale (KSS-J) within the driving simulator environment. The results showed that the gazing time has a significant statistical difference in relation to the drowsiness level: alert (1−5), weak drowsiness (6−7), and strong drowsiness (8−9), with P<0.001. Therefore, we suggested the potential of using the eye gaze to assess the drowsiness under a driving condition .


Introduction
A drowsy condition while driving is one of the leading causes of traffic accidents.In 1996, the National Highway Traffic Safety Administration (NHTSA) reported that driver drowsiness was involved in 1.2% to 1.6% of all police-reported crashes and 3.2% of the fatal crashes in the United States.The National Sleep Foundation estimated in 2002 that 51% of adult drivers had driven a vehicle while drowsy and 17% had fallen asleep behind the wheel.According to an estimation by the NHTSA in 2005, each year, 100,000 police-reported crashes are directly caused by drowsiness, which results in about 1,550 deaths, 71,000 injuries, and $12.5 billion in financial losses.The NHTSA also estimated that drowsy driving was responsible for 720,000 crashes, 44,000 injuries, and 800 deaths in 2013 [1].The National Police Agency of Japan released data showing that about 537,000 traffic accidents occurred in 2015.These accidents were caused by many factors such as unfocused driving, a lack of driving skills, and the physical condition of the driver (such as fatigue and drowsiness) [2].Some of the causes of these accidents, such as a lack of focus during driving and lack of driving skill, could probably be corrected.These shortcomings could be reduced by increasing driver awareness and driving skill.However, drowsiness, which is due to a physiological condition, is difficult to prevent.
Measurements of the physiological state have already been conducted in various studies.Previous studies have used brain wave [3−6], heart rate, muscle activity [7], and pulse rate [8], [9] characteristics to detect the drowsiness condition, and it was found that studies using electrodes attached to the human body yielded the maximum detection accuracy.However, those techniques are intrusive because the sensing elements (electrodes) used for measurement and required physical contact with the driver cause annoyance and discomfort during driving [10].Another appropriate method to detect and estimate the drowsiness state of a driver with less physical contact should be investigated.
Researchers have reported good results using numerous less intrusive techniques to detect the drowsiness of drivers, including eyelid movement and gaze or head movement monitoring [10].For example, researchers analyzed the amount of time that the eyelid was visible while driving in specific time intervals [11] using a camera placed in front of the driver.Investigations have often used the movement characteristics of the eye (eyelid movement) [12],  ISSN: 1693-6930 TELKOMNIKA Vol. 15, No. 2, June 2017 : 919 -925 920 the size of the pupil/eye [13], face monitoring to determine a change condition [14], and eye state analysis [15].These methods had the goal of providing information about the subject/driver while performing various tasks or under various conditions (e.g., resting state, fatigue, and drowsiness).However, limited results for the gazing-time properties have been reported.Even though this approach can detect the possibility of a drowsiness condition, its disadvantages include the need to provide a clear view and appropriate position for the camera.We believe that the use of a head-mounted device can overcome the view and position limitations while evaluating the driver's eye condition, especially to determine a drowsy condition [16,17].In a similar way, we also realized that only a limited number of drowsiness investigations have utilized head-mounted measurement techniques.Driving in a drowsy condition was examined by certain researchers [18,19] using different measurements.Even though the accuracy reached almost 99%, these results were obtained in sleep-deprived subjects rather than under a condition where natural drowsiness occurred.
In our study, we examined the use of a head-mounted eye tracker to track the eye's gazing time and assess drowsiness.As previously described, previous studies have commonly used a subject's eye and facial movement images to evaluate their condition, especially their gaze or focus on a certain position.However, we could not find any clear information about how to utilize the gazing time of the eye-gaze properties of the driver to determine a certain condition of the subject.To evaluate the drowsiness condition, we conducted a subjective evaluation of a subject's physiological condition using the Japanese version of the Karolinska sleepiness scale (KSS-J) to assess the drowsiness condition in relation to the experiment's location and environment [6].We divided the drowsiness condition into three levels, alert (KSS-J 1−5), weak drowsiness (KSS-J 6−7), and strong sleepiness (KSS-J 8−9), to estimate the early stage of drowsiness.We believe that these categories have a strong relationship with the eye-gazing properties, especially in relation to the human physiology point of view.We also investigated whether each feature of the gazing time properties had a significant statistical difference for the three levels.This study also examined the relationship between drowsiness and the physiological condition using the eye tracker.In particular, the gazing time signal was used as the actual parameter, and the KSS-J value was used as a subjective parameter to detect drowsiness in a driving simulator environment.

Subjects
Eight healthy males (21-35 years old) participated in this study.The subjects were asked to get enough sleep (at least 5 h) the night before the day of the experiment and eat lunch just before the experiment.They were also asked not to consume alcohol or caffeine prior the experiment.The subjects had to have a driver's license and gave informed consent before joining the research trial.

Experiments
The research environment of this research is shown in Figure 1.It consists of a driving simulator, eye gaze tracker, camera, computer to record the video, and driving simulator system control.The subjects were directed to travel on an oval track.A sunny noon condition was selected for the weather of the driving experiment, and there were no obstacles (traffic lights or another car passing) on the track.An automatic transmission was chosen for the driving configuration.
The experiments were conducted from 8:00 am to 10:00 am and from 1:00 pm to 3:00 pm (two experiments per day).Before and after each experiment, the subject was instructed to maintain a resting state in the driving simulator's seat, and the interviewer asked about their drowsiness condition every 30 s for 5 min.Then, the subjects performed simulated driving for 50 min, and a video of the subject's face was recorded.During this simulated driving, no sleepiness evaluation was conducted for the subject.A sleepiness evaluation was carried out after the subject finished the driving experiment by asking them to recall their drowsiness condition during driving.The evaluation was conducted by having the subjects watch their facial expressions captured by the camera and then asking them to provide a KSS-J score every 30 s.Each subject participated in eight trials of the same experiment on different days.

Psychological measurements
In this study, the KSS-J was used to evaluate the subject's drowsiness condition.KSS-J is a 9-level drowsiness questionnaire, in which the odd numbers represent the drowsiness degree, from very alert to very sleepy, as shown in Figure 3.

Analysis 2.4.1. Feature extraction
Figure 4 showed a representative example of the raw wave of the gazing time signal.A non-zero value for the gazing time (>0) means the occurrence of gazing.In contrast, a zero value for the gazing time (=0) indicates the existence of blinking or non-gazing.The number of occurrences of gazing, blinking, or non-gazing could be counted by frame.The continuous occurrence of each was sometimes seen and could be counted as a cluster.Therefore, the twelve features listed in Table 1 could be extracted from the gazing time signal and computed every 30 s.The process was repeated for all the features for each experiment.

Statistics
A Kolmogorov-Smirnov test was also used to examine whether the gazing time signal had a normal distribution before performing a statistical analysis.A one-way ANOVA analysis was used if the distribution data had a normal distribution, and a Wilcoxon-rank sum analysis was used if the distribution data did not have a normal distribution.The results of the feature extraction stage were divided into three categories: KSS-J 1−5, KSS-J 6−7, and KSS-J 8−9.After dividing the gazing time signal into three categories, a statistical analysis was performed to investigate the significant differences within these three categories.A value of P < 0.05 was considered to be statistically significant.

Results
Table 2 lists the typical statistical results of all the features.We can see that the gazing frame (GF), gazing cluster (GC), total gazing time (TGT), and non-gazing cluster (NC) occurrences decreased when the subject became drowsy, and the statistical test showed a significant difference (P < 0.001; Kruskal-Wallis test) regarding the differences in each category followed by a decline in the parameter trend for the drowsiness condition.In contrast, the blink frame (BF) occurrence, blink cluster (BC) occurrence, ratio of BF/BC (RB), and ratio of NF/NC (RN) increased when the subject became drowsy, and the statistical test also provided these differences (P < 0.001; Kruskal-Wallis test).In another case, even though the blink cluster (BC) occurrence, gazing frame density, gazing cluster density, and ratio of GF/GC (RG) had the tendency to change with the drowsiness condition, these parameters were not considered to have a significant difference in all the drowsiness categories.Moreover, the non-gazing frame (NF) occurrence did not shown any tendency in relation to the drowsiness condition because it had inconsistent results.
Regarding the statistical results, we obtained information about both increasing and decreasing trends for the parameters representing the drowsiness condition, especially for class 2, which describes the state before becoming drowsy.Thus, the results showed that the gazing ISSN: 1693-6930  Gazing Time Analysis for Drowsiness Assessment Using Eye… (Arthur Mourits Rumagit) 923 time parameters could be used to estimate the state before becoming drowsy.However, several subjects did not show a significant difference in relation to the physiological measurements using the KSS-J.This was attributed to differences in perception during the subjects' evaluations when measuring and recalling their physical state during driving when watching the video recording of their driving.

Discussion and Conclusion
Drowsiness is considered to be related to the human physiological condition.The simplest way to determine the physiological state of a human body is by directly asking the subject about their present condition or having them fill out a drowsiness assessment questionnaire.Some researchers have used the Karolinska sleepiness scale (KSS) in questionnaires to obtain the physiological condition, especially the drowsiness condition of a subject [3,6].Moreover, one of the previous studies evaluated the performance of the KSS in drowsiness evaluation research.It was concluded that an increase in the KSS value represents the condition of the subject becoming drowsy.We used an eye tracker mounted on the head to observe the drowsiness condition during driving using a real driving simulator.We extracted the gazing time features from the eye-tracker device, and analyzed the KSS-J properties as our evaluation drowsiness method.In this study, we found that several features of the gazing time had statistically significant differences with three drowsiness levels: alert (KSS-J 1−5), weak drowsiness (KSS-J 6−7), and strong drowsiness (KSS-J 8−9), with P < 0.001.We utilized the features of the gazing frame (GF) occurrence, gazing cluster (GC) occurrence, blink frame (BF) occurrence, nongazing cluster (NC) occurrence, ratio of blinking frame versus cluster (RB = BF/BC), ratio of non-gazing frame versus cluster (RN = NF/NC), and total gazing time (TGT).Overall, these seven gazing time features were able to detect the drowsiness condition and could be used as new phenomena in drowsiness studies using an eye tracker mounted on the head to obtain the characteristics of the state of the eye.We believe that these combinations effectively represented the physiological aspects of the eye properties and could be used to determine the drowsy condition.The eyes are commonly known to be a part of the human body that can clearly represent the human condition of being asleep or awake.In particular, the pupil size and its pupillary waves are strongly associated with alertness, drowsiness, and the sleep condition.From a physiological point of view, the pupillary behavior is hidden or distorted by the closing of a human's eyelids when we become drowsy or fall asleep [20].Using the gazing time properties, we can generally say that a human's eyes easily become unfocused while starting to fall asleep or becoming drowsy.In contrast, a human's gaze is focused when concentrating on a specific object.By evaluating the eye-gaze properties representing several drowsiness level conditions, classification techniques can also be used to quantify the performance of the parameters.It has recently been shown that machine learning or classification methods such as the support vector machine could be used to classify the drowsiness condition with an accuracy of up to 99% in a driving simulator environment [12].However, this study employed sleep-deprived subjects, performing the experiment on the morning following the subjects lack of sleep.A neural network has also been used as a classification method to detect drowsiness [19].Regarding the previous studies results, we should consider using a gazing time parameter to test the drowsiness detection performance using a classification method or machine learning in the next step of our investigation.By considering specific phenomena, we obtained useful parameters using the gazing-time properties and showed that using the ratio of different parameters provided significant differences between subjects as well as the gazing time features themselves.The use of combinations of specific parameters showed that gazing time parameters composed of several features could improve the estimation of drowsiness.
ISSN: 1693-6930 Gazing Time Analysis for Drowsiness Assessment Using Eye… (Arthur Mourits Rumagit) The gazing time signal was obtained using the eye gaze tracker, as shown in Figure2.The signals were sampled at 30 Hz.

Figure 4 .
Figure 4. Representative example of raw wave of gazing time signal

Table 2 .
Typical statistical results of all features