• Author / Uploaded
• Aman

Citation preview

Design of a Vehicle Driver Drowsiness Detection System through Image Processing using Matlab Melissa Yauri-Machaca Image Processing Research Laboratory (INTI-Lab) Universidad de Ciencias y Humanidades Lima, Perú [email protected]

Brian Meneses-Claudio Image Processing Research Laboratory (INTI-Lab) Universidad de Ciencias y Humanidades Lima, Perú [email protected]

Natalia Vargas-Cuentas Professional Member, IEEE Image Processing Research Laboratory (INTI-Lab) Universidad de Ciencias y Humanidades Lima, Perú [email protected]

Avid Roman-Gonzalez Senior Member, IEEE Image Processing Research Laboratory (INTI-Lab) Universidad de Ciencias y Humanidades Lima, Perú [email protected]

Abstract— A person when he or she does not have a proper rest especially a driver, tends to fall asleep causing a traffic accident. It is why the present work wants to realize a system that can detect the drowsiness of the driver, in order to reduce traffic accidents. For that system, it will take the processing of images through a camera which will focus on the driver. In that, it is going to analyze the changes that happen in the face and then will be processed through a program in order to detect drowsiness to send an alert to the driver. Keywords— Drowsiness, system, processing of images, face.

I.

INTRODUCTION

The human has a fundamental need called sleep because with adequate rest, helps to maintain efficient operation of the capacities that realize a person. However, when this activity is interrupted by several factors, especially in a driver, it deteriorates the psychomotor and cognitive functions such as reaction time, the capacity of surveillance, judgment, and attention. When the driver does not have adequate rest, the driver will try to sleep while driving and this is the main sign of drowsiness. In most cases, the driver does not pay attention, and then the yawns come, the attempt to close the eyes every moment and the movement of the head from side to side [1]. It has as a consequence a traffic accident, which is an unfortunate event for the driver of the vehicle, as well as for the pedestrian. Currently, there are studies for the creation of a drowsiness detection system, which extracts the essential characteristics of drowsiness of the driver to determine the level of drowsiness. In these systems, which are controlling the state of drowsiness of the driver through a webcam with night vision to track the driver in real time and when drowsiness is detected, the system will send a warning [2]. The objective of these systems is to improve the safety of people with the use of detection and alarm to avoid accidents caused by drowsiness of the driver and which are harmful to the users of the route [3]. In Peru, according to the Instituto Nacional de Estadística e Informática shows that 0.6% of traffic accidents are caused

978-1-5386-6122-2/18/$31.00 ©2018 IEEE

by drowsiness because the driver believes that taking a rest for a few seconds, closing his eyes, it can cause to losing the control of the vehicle [4]. For this reason, in the present investigation, the designs of a system able to detect the drowsiness of the driver in order to be alerted are presented. Also, drowsiness detection of the driver's approach driver is confronted with image processing to recognize drowsiness patterns. In section II, the development of the objectives. Then, in section III presents the results according to the objectives. In section IV, a discussion about the results is shown and, finally, in section V, which exposes the conclusions. II.

METHODOLOGY

The development of the research is following the next objectives below: A. The causes of driver drowsiness •

Sleep less than 8 hours: According to a study carried out in the final bus station of Huancayo to 100 interprovincial bus drivers, it states that in the 24 hours prior to the survey, 47% of the drivers had slept six hours or less [5]. This sample, the driver does not obey the eight hours appropriated for the quality of sleep in order to affect the health. For example, the person gets sick every moment, decrease in the mood and quick reaction to avoid some inappropriate event.

In addition, in [6] developed a study on the number of hours that the driver sleeps in a formal and informal business. This study focuses on day and night and shows in tables 1 and 2.

Therefore, the largest percent of the sleeping environment is in the trunk of the car. This causes various sleeping problems such as body ache and is not an appropriate place due to the existence of noise from other cars. So the person recognizes that the noise damages the quality of sleep.

Fig.1: hours of sleep in formal and informal drivers on day shift

• No work schedule: Drivers realize long work days without systematic programming does not allow adequate rest. They work in a disorganized way on day shifts and night shifts and rest an average of four to five hours per day [6]. On the other hand, the driver's job does not end when they arrive at their destination, because they have to clean the car. And in [7] according to Table 4 shows an alarming number of days the driver works. 20% in the formal business and 29% in the informal companies, where the drivers work every day. Fig. 4: work shifts by night shift per week

Fig. 2: Hours of sleep in formal and informal drivers on night shift

In Table 2, there is not much difference between informal and formal drivers in the night. The study presents 42% of formal drivers and 30% of informal drivers sleeping around 5 to 7 hours. This happens when the drivers modify the circadian rhythm because the human is destined to perform some action in the morning and at night is sleeping for the hormone melatonin. And for that reason, the driver cannot sleep well. • There is no appropriate sleep environment: The driver has to make a change with another driver for rest. But the sleep environment does not have the necessary implements, nor the conditions of space and calm to sleep. In [5] it describes a study at the bus station Fiori and Huancayo. The first has 81% of drivers sleep in the trunk of the car and in the second is 62%. In addition, in the first final bus station, 50% of the drivers sleep when the car is in motion and 42% of the final second bus station. In table 3 shows what has been described.

B. The patterns of drowsiness The drowsiness in the driver presents several facial changes such as: •

Frequent flicker

• •

Moving the head from side to side Yawn

In table 5, a study is shown in Arequipa about the company Corattsa and other companies. In this study, the common patterns of drowsiness in the driver are detailed.

Fig. 5: Sleepiness patterns

Fig. 3: Driver's sleep environment

In the study of the first route is towards Cerro Verde, the second is towards the province of Arequipa and the third route are provinces adjacent to Arequipa and the fourth route is from Arequipa to Lima. According to the image, the most relevant model is to open and close the eyes (blink), which has 59% in

the third route [8]. And this pattern will be analyzed to detect drowsiness. Blinking is a small eyelid depression whose main purpose is to keep the external part of the eye moist, avoiding the evaporation of the tear film and maintaining the integrity of the ocular surface and the optical quality of the cornea [9]. The importance of opening and closing the eyes, offers good vision because a driver with drowsiness has blurred vision. In drowsiness opening and closing the eyes is more frequent and does not fully realize. Since the blinking is very fast and does not close the eyelids. In the blink of an eye, two factors that influence it will be studied, which are: • Flicker frequency: The number of blinking that a driver makes during a certain time. According to equation 1. °



1

The blinking frequency in a person with sleep is approximately 21 blinks per minute and in the normal state, the person has 15 blinks per minute. • Opening of the eyes: It is the amplitude between the two eyelids at the time of opening and closing the eyes in the closing stage, as shown in image 6. The process begins when the pupils of the eyes are covered by the eyelids. that the upper and lower eyelids are open.

Fig. 6: The closing process of a blinking

And for the calculation of this factor PERCLOS is used, to determine the percentage of the closing of the eyelids. The following mathematical formula presented in equation 2 is used, where t1 to t4 is the time by which the opening of the eye will be limited to being completely open from the closure. 100% 2

C. The stages of the system In this objective, each part of the driver's drowsiness detection system is developed, which consists of the stage of image acquisition, processing, detection and warning. These pieces are [10]: • Image acquisition: In this stage, the camera is used to acquire the image of the driver. In the camera, an adjustment is made in the center of the camera as shown in figure 7. The axis moves around 90 ° to capture a good image and when it is necessary to use the lighting built into the camera.

Fig. 7: Camera Configuration

In this stage, which consists in connecting the camera with the MATLAB software, so it emits a command called "imaqhwinfo". This command permission is to know the name of the camera in the software, in this case, it is called "WinVideo". And, finally, the "imaqhwinfo ('WinVideo', 1)", the command is executed to check the characteristics of the connected camera, as well as the resolution. For the capture of the images, two environments are chosen, the first is considered in the day and the second in the night. At the time of image acquisition, it focuses on the driver's face. In the software, an algorithm is developed generating a frame with a rectangle to the face, which indicates that there is a face to be processed as shown in figure 8. This process is done to more effectively produce the detection of the patterns of drowsiness.

Fig. 8: Face Detected

• Image Processing: In this stage, the image becomes a two-dimensional matrix when it is processed in MATLAB as shown in figure 9. Each element of the matrix corresponds to a pixel of the image. The fundamental thing is the preparation of the image so that the detection is effective. The process begins with the modification of the characteristics of the image to achieve an improvement in the image. The characteristics are: the contrast and the noise that exists in the image. On the other hand, several filters are applied to the image before detection.

Fig. 9: Matrix of the image

• Detection: At this stage, we will proceed with the identification of drowsiness patterns. To process the patterns will begin with the isolation of the section of interest as the eyes. Then, the process of extracting characteristics begins, which analyzes the moment of closing the eyes and the distance from the opening of the eyes. As shown in Figure 10, the programming takes 3 frames where the first one will capture the eyes of the driver, in the second frame, it places the eyes and finally, in the third frame the program analyzes that the eye is opened by a green circle.

Fig. 11: Image processing with close eyes

•

Alarm: Finally, after detection, the system will emit an audible alarm to warn the driver to have drowsiness. The alarm varies according to the pattern that has been detected in relation to the sensitivity level. So the driver does not get used to a repetitive tone and ignores the warning.

The system consists of the camera, which acquires the image. The camera used belongs to the brand Micronics with the model W360 MIC and consists of high resolution with great clarity and is connected via USB to the laptop. The use of the laptop will proceed with image processing and driver drowsiness detection using MATLAB software. Finally, the alarm is integrated into the software. The system is in figure 12.

Fig. 12: System elements

The MATLAB program is showing next:

Fig. 10: Image processing with open eyes

Figure 11 shows the same process, but in this case with closed eyes indicating that there is a blinking.

clear all clf('reset'); cam=webcam(); %camara parpadeo=imread('hay_parpadeo.jpg'); noParapadeo=imread('no_parpadeo.jpg'); detector = vision.CascadeObjectDetector(); detector1 = vision.CascadeObjectDetector('EyePairBig '); contador =0; while true vid=snapshot(cam); % vid = rgb2gray(vid); img = flip(vid, 2); %

bbox = step(detector1, img); if ~ isempty(bbox) biggest_box=1; for i=1:rank(bbox) %find the biggest face if bbox(i,3)>bbox(biggest_box,3) biggest_box=i; end end faceImage = imcrop(img,bbox(biggest_box,:)); % stop the face bboxeyes = step(detector1, faceImage); % eyes located subplot(3,2,1),subimage(img); hold on; % showing for i=1:size(bbox,1)

floor(r+r/2)], 'ObjectPolarity','dark', 'Sensitivity', 0.93); % Hough transform [M,I] = sort(radii, 'descend'); eyesPositions = centers;

subplot(3,2,2),subimage(eyesImage); hold on; viscircles(centers, radii,'EdgeColor','g'); %% count

cent= numel(eyesPositions); if cent ==0 ; contador= contador+1; disp('hay parpadeo') subplot(3,2,4); subimage(noParapadeo); else cent==1; contador =contador; disp('no hay parpadeo'); subplot(3,2,4); subimage(parpadeo);

% focus

the face rectangle('position', bbox(i, :), 'lineWidth', 2, 'edgeColor', 'y'); end subplot(3,2,3),subimage(faceImage); if ~ isempty(bboxeyes) it eyepair is available

%check

biggest_box_eyes=1; for i=1:rank(bboxeyes) %find the biggest eyepair if bboxeyes(i,3)>bboxeyes(biggest_box_eyes, 3) biggest_box_eyes=i; end end

bboxeyes=[bboxeyes(biggest_box_eyes,1),b boxeyes(biggest_box_eyes,2),bboxeyes(big gest_box_eyes,3)/3,bboxeyes(biggest_box_ eyes,4)]; %resize the eyepair width in half eyesImage = imcrop(faceImage,bboxeyes(1,:)); %extract the half eyepair from the face image eyesImage = imadjust(eyesImage); %adjust contrast

r = bboxeyes(1,4)/4; [centers, radii, metric] = imfindcircles(eyesImage, [floor(r-r/4)

end if contador >= 5; disp('Tienes somnolencia'); end end end end set(gca,'XtickLabel',[],'YtickLabel',[]) ; hold off; As shown in the program, it has a limit of 5 blinks for a period of time, in case it is exceeded, an image is shown indicating that it has exceeded the number of blinking norms. III.

RESULTS

The analysis of the causes of drowsiness, the axis of the causes lies in that it is not a quality of sleep, which is dominated by the hours the driver sleeps, the place where the driver and there is not a correct work schedule. Then in the second objective, the result is that the pattern of drowsiness is in the eyes because drowsiness does not allow the eyes to be kept open. Finally, it was obtained that the most notable characteristics are the frequency of blinking and the opening of the eyes. The design of the system is based mainly on the analysis of the patterns of these when driving so, when

the driver shows these variations on his face, which is detected. The result of the study of the stages of the system, you get the hierarchical knowledge of how the system will work, this information is important because it will have to be declared hierarchically in the Matlab software. Also for each stage the necessary characteristics were obtained to make adjustments when processing the image. IV.

DISCUSSION

The present study confirms that drivers do not have adequate rest, which causes drowsiness. And drowsiness is one of the factors that cause traffic accidents. Therefore, it is necessary to perform a system that detects the drowsiness of the driver. There are several studies, where the system detects drowsiness of the driver of the vehicle, the drowsiness patterns are such as: blinking, level of distraction and yawns. These patterns determine a level of drowsiness however the level of distraction presents difficulties in the case of this outside a range established to detect the face. In this case only in the frequency of blinking and the opening of the eye since they are the most dominant symptoms that fit the camera within the rectangle that is set to detect the face, which helps to have more efficiency in the system. Also in this study as in others, it is important to order the stages of the system as this verifies the elements that are used and the requirements that are needed to make the system more efficient. In conclusion, the analysis of drowsiness patterns using image processing is innovative due to the detailed analysis of facial changes, that is, they have a real-time analysis of what the driver experiences when drowsiness enters. V.

CONSLUSIONS

In conclusion, drivers have a bad habit to sleeping anywhere, which provides a bad break and as a consequence they have drowsiness.

In summary, drowsiness patterns are a fundamental part of the detection process, as this alerts the driver to reduce traffic accidents In summary, each stage of the system depends on the other, therefore, it is important to know what stages, our system will have. Knowing the stages, we can know the hierarchy of the installation or operation. REFERENCES [1] [2] [3] [4]

[5]

[6] [7]

[8]

[9] [10]

R. Jimenez Moreno, «Sistema de detección de nivel de cansancio en conductores mediante técnicas de visión por computador,» Universidad Nacional de Colombia , Bogotá , 2011. G. D. Brito Moncayo y W. L. López Romero, «Sistema de control del estado de somnolencia en conductores de vehículos,» Universidad Técnica de Ambato, Ambato, 2016. K. Saldaña Altamirano y R. Rodríguez Urquiaga, «Detección de la somnolencia usando maquinas de soporte vectorial a partir de,» Universidad Nacional De Trujillo, Trujillo, 2014. Instituto Nacional de Estadistica e Informatica, Noviembre 2015. [En línea]. Available: https://www.inei.gob.pe/media/MenuRecursivo/publicaciones_dig itales/Est/Lib1308/libro.pdf. [Último acceso: Mayo 10 2017]. E. Rosales Mayor , I. S. Durand Vila , N. E. Montes Ccaccro , C. . G. Alonso Cueva , L. L. Merino Baquerizo , . M. J. Rey de Castro , M. T. Egoavil Rojas Martha Teresa, R. E. Flores Herrera y S. L. Rivera García , «Accidentes de carretera y su relación con cansancio y somnolencia en conductores de ómnibus,» Revista Medica Herediana, vol. 20, nº 2, pp. 48-59, 2009. E. Rosales Mayor y J. Rey De Castro Mujica, «Somnolencia: Qué es, qué la causa y cómo se mide,» Acta Médica Peruana, vol. 27, nº 2, pp. 137-143, 2010. G. Liendo, C. Castro y J. Rey de Castro, «Cansancio y somnolencia en conductores de ómnibus interprovinciales: estudio comparativo entre formalidad e informalidad,» Revista Peruana de Medicina Experimental y Salud Publica, vol. 27, nº 2, pp. 187-194, 2010. R. Castro y S. Soriano, «Hipersomnia durante la conducción de vehículos ¿causa de accidentes en carreteras? A propósito de un estudio cualitativo.,» Rev. Soc. Peru Med Interna, vol. 15, p. 142, 2002. N. Quevedo López, « Estudio del parpadeo durante la conducción de vehículos (aspectos cognitivos y de flujo de información),» Universidad Politécnica de Catalunya., Barcelona, 2012. Vargas-Cuentas, Natalia I., and Avid Roman-Gonzalez. "Facial image processing for sleepiness estimation." Bio-engineering for Smart Technologies (BioSMART), 2017 2nd International Conference on. IEEE, 2017.