IRJET- Drowsiness Detection of Drivers using IoT Image Processing and Machine Learning Techniques

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 07 Issue: 08 | Aug 2020

p-ISSN: 2395-0072

www.irjet.net

DROWSINESS DETECTION OF DRIVERS USING IOT IMAGE PROCESSING AND MACHINE LEARNING TECHNIQUES Susheelamma K H1, Smriti Gururaj2 1(Asst.Professor,

ISE, SJC Institute of Engineering and Technology, India) Student,, ISE, SJC Institute of Engineering and Technology, India) ---------------------------------------------------------------------***---------------------------------------------------------------------2(BE

Abstract - Driver drowsiness is one of the most widely recognized purposes behind the street mishaps .Rest issues causes the weariness of driver and this outcomes in the expanding number of street mishaps. These days about thirty percent of street mishaps are caused because of the sharpness of the driver. So as to decrease the mishaps identified with the driver languor it is important to outline a framework which precisely identifies the sleepiness of the driver .By recognizing the tiredness level and giving an admonition to the driver the mishaps can be evaded. This paper portrays a strategy which distinguishes the driver weakness precisely continuously and to give a notice signal. In this paper Raspberry pi 3 module is utilized to distinguish whether the driver is exhaustion or not utilizing EAR strategy. Raspberry pi is utilized on the grounds that it is quicker to recover pictures from the camera and sufficiently quick to identify a driver's element progressively. In this paper a method to recognize driver drowsiness utilizing of Open CV (open source PCvision), Raspberry pi and picture handling is introduced.

Key Words: Drowsiness, Open CV, Face detection, Eye detection, HOG, EAR.

1. INTRODUCTION The consideration level of driver debases due to less rest, long consistent driving or some other ailment like mind issues and so forth. A few reviews on street mishaps say that around 30 percent of mishaps are brought about by weariness of the driver. At the point when driver drives for more than typical period for human then over the top exhaustion is caused and furthermore brings about sleepiness which drives the driver to tired condition or loss of cognizance.

Laziness is an intricate wonder which expresses that there is a diminishing in cautions and cognizant levels of the driver. Despite the fact that there is no immediate measure to recognize the tiredness however a few backhanded strategies can be utilized for this reason.

There is a developing requirement for a shrewd and practical tiredness identification framework that the various cars in the business can rapidly adjust. The fields of AI and fake knowledge have made various weighty advances, which utilize various calculations to train the model to be brilliant and self-governing. This model proposes one such calculation, known as the Haar Cascade calculation, which is joined with the different libraries in Python to catch and distinguish languor continuously.

2. LITERATURE SURVEY Ovidiu Stan et.al. Says in the paper [1] "Eye-Stare Tracking Method Driven by Raspberry PI Pertinent in Automotive Traffic Safety" that This paper comes as a reaction to the way that, of late, an ever increasing number of mishaps are brought about by individuals who nod off at the worst possible time. Eye following is one of the most significant angles in driver help frameworks since natural eyes hold much in- development with respect to the driver's state, similar to consideration level, look and weariness level. The occasions the subject squints will be considered for distinguishing proof of the subject's tiredness. Additionally the bearing of where the client is looking will be assessed as per the area of the client's eye stare. The created calculation was executed on a Raspberry Pi board so as to make a convenient framework. The principle assurance of this venture is to imagine a functioning eye following based framework, which centers around the languor discovery among weakness related lacks in driving.

Kulkarni S. S. et.al. Says in the paper [2] "Utilization of raspberry pi based implanted framework for constant insurance against street mishaps because of driver's drowsiness and additionally alcoholic and drive cases" that Present work manages the utilization of raspberry pi CPU based detecting framework to the discovery of driver's dormancy and liquor abuse so as to maintain a strategic distance from the street mishaps. The implanted framework comprises of 5-megapixel advanced camera, liquor location sensor and the signal interfaced to the microcontroller.

The inserted framework is constrained by Raspbian working framework. The framework recognizes ongoing circumstance of the driver's cautiousness and power over the vehicle. In the event that drunkard and/or languor tests are certain, it turns on the caution, (ii) turn off the vehicle's motor by means of microcontroller- based program

Š 2020, IRJET

|

Impact Factor value: 7.529

|

ISO 9001:2008 Certified Journal

|

Page 3592

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 07 Issue: 08 | Aug 2020

p-ISSN: 2395-0072

www.irjet.net

controlling start power source and (iii) sends a SMS to the individual near the driver's area.

Vehicle based measures is one of the strategy which is utilized to gauge driver laziness. This is done by setting sensors on various vehicle parts, which incorporate controlling haggle increasing speed pedal. By investigating the signs from the sensors, the degree of tiredness can be decided. Normally utilizing vehicle- based measures for identifying the degree of driver sleepiness are the controlling wheel development and the standard deviation in horizontal position. A guiding point sensor which is mounded on the directing of vehicle is utilized to quantify the controlling wheel development. The quantity of smaller scale redresses on the controlling wheel diminishes contrasted with typical driving when the driver is sluggish [1]. In view of little SWMs of somewhere in the range of 0.5° and 5°, it is conceivable to decide if the driver is sluggish and along these lines give a caution if necessary [2].

Another vehicle based measure usedto gauge the languor of driver is SDLP. Here the position of path is followed utilizing an outside camera. The principle constraint of this technique is that it reliant on outside variables, for example, street markings, lighting and climatic conditions. Along these lines, these driving execution measures are not explicit to the driver's laziness[3].

Physiological estimates are the target proportions of the physical changes that happen in our body in light of weakness. These physiological measures can be used to gauge the weariness level and can give caution to the drivers.

These physiological changes can be essentially estimated by separate instruments, for example, electrocardiogram (ECG), electrooculography (EOG), electroencephalography (EEG) and electromyogram (EMG). Electrocardiogram is one of the physiological estimates which can be used to gauge the exhaustion of driver. Here ECG anodes are utilized to gather ECG signals from the body which gives the basic boundaries identified with Heart Rate (HR), Heart Rate Variability (HRV) and breath rate or relaxing recurrence. Each of these are identified with laziness [4].

Electroencephalography (EEG) is one of the most dependable physiological measures for tiredness identification.

EEG anodes are set at right place and get information from cerebrum. In the wake of pre processing the information, which is procured from the EEG anodes can be separated into various recurrence groups. The pre processing includes ancient rarity expulsion and sifting. Usually utilized recurrence groups incorporate the delta (0.5– 4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–30 Hz), and gamma (more prominent than 30 Hz) groups [5]. Force range of EEG cerebrum waves is utilized as a pointer to distinguish the tiredness of the driver. Here, EEG intensity of the alpha, theta groups increments and the intensity of the beta groups diminishes. The EEG based languor discovery isn't effectively implementable. Since the driver needs to wear an EEG top while driving a vehicle. These gadgets are being distractive and this is the fundamental hindrance of this strategy.

Conduct changes occur during drowsing like yawning, measure of eye conclusion, eye squinting and so on. In typical condition the pace of yawning will be less. At the point when the driver is in weariness the pace of yawning will be far higher than the typical. So by watching this yawning rate we can distinguish whether the driver is in weariness or not [5].

In eye conclusion strategy the tally of eye squint of the driver is estimated for getting the state of the driver. The normal length of a typical eye flicker is 0.1s to 0.4s. That implies, in one second the eye will flicker in any event 2 or multiple times.

This is watched for a couple of moments. At the point when the driver is in exhaustion the check will be far less thought about to the typical condition. Subsequently we can recognize whether the driver is in weakness or not [6]. The primary strategies utilized for eye squint identification are Eye Aspect Ratio (EAR) technique and Template Coordinating strategy.

The Ear technique is finished by computing an amount named EAR. In ordinary condition the estimation of EAR is practically consistent. On the off chance that the driver is in exhaustion the EAR worth will be roughly close to zero.

Subsequently we can recognize whether the driver is in weakness or not. In layout coordinating strategy, the identified eye is changed over to a format. Furthermore, this format is put away.

Each casing that we have gotten is coordinated with this format. In this way we get a relationship esteem as a lattice. In the event that the connection esteem is high the relating outline is entirely coordinated with the limit format. Also, on the off chance that this worth is less, at that point the edge isn't coordinating with the layout. Subsequently we can distinguish whether the driver is in weakness or not [6- 7].

© 2020, IRJET

|

Impact Factor value: 7.529

|

ISO 9001:2008 Certified Journal

|

Page 3593

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 07 Issue: 08 | Aug 2020

p-ISSN: 2395-0072

www.irjet.net

3. EXISTING METHOD A significant factor for causing mishaps in rush hour gridlock that the driver's exhaustion. Numerous nations are occupied with research here effectively now. Instructions to successfully screen and forestall driver weakness driving have a lot of genuine centrality to decrease auto collisions and faculty mortality.

At the point when the driver is exhausted, the head will in every case descending slanting. As indicated by measurements, the connection coefficient of head position and weakness degree is about 0.8. Be that as it may, some driver's head stance won't change essentially, the connection coefficient will be negative and the framework's adjudicator and early admonition fizzled.

Framework identifies the driver's psychological state by checking guiding wheel's developments and examples. With the extending of the driver weakness, the quantity of significantly pivot will be increment; the handgrip quality will expand.

This technique screens the time and the deviation level of vehicles leaving from the white lines by introducing camera in a similar viewpoint with the driver on the vehicle. This estimation requires the white line must be exist and clear enough out and about, so the impedance of outside conditions is incredible.

The EEG isn't on the exhibition of a similar when the cerebral cortex is in fervour or hindrance. As per the EEG's recurrence dispersion and waveforms, expect the capacity status of the cerebrum action, to guess whether the driver is exhausted. Anyway the EEG is defenceless against impedance from outer components and there are such a significant number of contrasts in individual physiological reaction.

4. METHODOLOGY The various viewpoints and highlights gave by the framework to recognize languor just as different highlights to give interruption location are clarified.

Figure 1 shows the flowchart of the tiredness discovery framework. As referenced, the procedure starts with social occasion surges of information by gathering picture outlines from the video real time by means of the camera module. The gathered pictures are utilized for preparing, and the drowsiness is identified. When drowsiness is recognized, the individual driving vehicle is alarmed in the vehicle through the speakers in the vehicle just as in the portable application.

4.1 OVERVIEW DESIGN An IOT cum Android based venture where we guarantee the wellbeing of vehicle drivers by distinguishing sleepiness, recognizing mishap telling all the close by medical clinics and GPS following where proprietors can follow their vehicle significantly in the wake of being taken through their application. Utilizing Raspberry PI3 and a typical PI camera module (perfect with RPI), through python open CV we distinguish whether a driver is feeling drowsy or not by following his eyes. Utilizing GPS module, GSM modem and accelerometer sensor we can distinguish at whatever point a vehicle is engaged with mishap. For android application, we have included ordinary information exchange login page alongside Firebase-auth sign-in page. At whatever point mishap is identified by our rpi3, it will advise our application through firebase and afterward our application will send messages with the GPS areas to the medical clinics.

4.2 FACE DETECTION Driver Drowsiness Detection requires a video sensor to distinguish the essences of drivers. A Pi Camera was utilized in this limit. Likewise, it is prepared by Raspberry Pi. This camera is additionally used to identify if there is any sort of interruption in the framework.

This specific camera will first identify the face and contrast it and the format accessible in the database with the concerned power. On the off chance that the face matches with the given format, at that point there is no sort of interruption recognized. On the off chance that the face doesn't coordinate the layout the concerned power will get a warning and interruption occurring.

Š 2020, IRJET

|

Impact Factor value: 7.529

|

ISO 9001:2008 Certified Journal

|

Page 3594

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 07 Issue: 08 | Aug 2020

p-ISSN: 2395-0072

www.irjet.net

4.3. EYE DETECTION The camera can identify the substance of the driver; we ascertain the tiredness level of the driver in light of eye flicker rate. For example, if drivers flicker their eyes all the more every now and again, the creators expect that the drivers are lazy. Along these lines, it is fundamental for this paper to recognize eyes precisely so as to ascertain for eye squint recurrence. The camera perceives the contrast between the ordinary eye squint and a sluggish eye. This aids thus in lessening the bogus alert with respect to the same.

Fig 2: Normal person without drowsiness output In Figure 2, the eyes of the user are in normal state and the eye blinking rate is less than expected. So it doesn’t give any false alarm about the drowsiness of the driver.

Fig 3: Person detected with drowsiness

Š 2020, IRJET

|

Impact Factor value: 7.529

|

ISO 9001:2008 Certified Journal

|

Page 3595

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 07 Issue: 08 | Aug 2020

p-ISSN: 2395-0072

www.irjet.net

In this specific yield, the eyes of the client stay shut for more than the necessary edges every second and subsequently the languor level is distinguished. This is presently told to the concerned position to caution the client.

4.4 HARR CASCADE CLASSIFIERS In Haar Cascade Classifiers, a great deal of comparative and disparate pictures is prepared so as to identify weariness of the driver. Open CV is a learning- based strategy, pressed with an identifier just as a coach. For preparing, a different database is kept up for face and eye with a few positive and negative pictures having eye shut and opened conditions and diverse set facial pictures [15]. In 2013, Patil et al recommended a dream-based sleepiness is completed utilizing Support Vector Machine and Haar Cascade Classifiers [14].

4.5 SHAPE PREDICTOR_68_FACIAL LANDMARK IDENTIFICATION AND EYE ASPECT RATIO [EAR] So as to anticipate the face and eye area in the live video transfer, shape indicator is utilized. Fig.1 shows the drowsiness which is estimated by computing the eye perspective proportion (Euclidean separation between the eyes are determined), the contentions are passed to the Predefined dataset and facial milestone recognition is completed. For each video arrangement, the eye milestones are found. The angle proportion among width and stature of the eye is aligned.

Fig 4: Close and open eyes with landmark pi detected automatically by [1]. The EAR is calculated for several frames of a video. A single blink is represented.

Š 2020, IRJET

|

Impact Factor value: 7.529

|

ISO 9001:2008 Certified Journal

|

Page 3596

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 07 Issue: 08 | Aug 2020

p-ISSN: 2395-0072

www.irjet.net

Fig 5: Drowsiness detection EAR = ||�2–�6|| + ||�3–�5|| 2||�1 – �4|| Where p1,‌,p6 are the two-dimensional milestone area, spoke to in Fig.2. The EAR is for the most part stable when an eye is open and is drawing near to zero while the eye isn't in open state. On the off chance that the individual review the camera Ceaselessly ,the Eye Aspect Ratio(EAR)is seen as typical and it arrives at low worth when he/she shutting the eye for a more extended time. At the point when the lower esteem is reached, at that point drowsiness is distinguished. In 2012, Ubidots expounded on associated programming and equipment answers for distantly control, computerize forms for social insurance customers and screen. Twilio is a cloud correspondence stage as a help (PaaS), it permits a product designer to automatically send and get instant message utilizing its web administration APIs, make and get calls. The paper presents cautioning process when the driver feels sluggish utilizing cloud worker and portable API to communicate something specific and simultaneously giving an alert sign to the driver.

5. FUTURE SCOPE A non-meddlesome framework to confine the eyes and screen exhaustion was created. Data about the head and eyes position are gotten through different self-created picture preparing calculations. During the observing, the framework can choose whether the eyes are opened or shut.

At the point when the eyes have been shut for two seconds, an admonition signal is given. Likewise during observing, the framework canconsequently distinguish any eye limiting mistake that may have happened. If there should arise an occurrence of this sort of mistake, the framework can recoup and appropriately restrict the eyes.

The proposed framework was tried on the genuine driver pictures. The video picture [480 x 640 pixels] of 75 diverse test people has been recorded during a multi day, night and complex foundation at better places. The proposed framework has two key stages, for example, pre processing also, distinguishing eye from video pictures.

In pre processing, new improved strategy is utilized to improve the differentiation of dull areas and tried with existing calculation. According to the outcomes, all the commotions in the video picture are expelled effectively. In second stage new methods are utilized to extricate eye from the pre processed picture. It is additionally tried with standard existing strategy and the correlation results. The eye pair can be chosen effectively as a rule. The bogus finding pace of drowsiness of Colour signal and projection capacity, BTMED and KDKBM is 21.7%, 14.7% what's more, 10.3%. Since the bogus discovering rate is just 10.3%, the proposed framework is successful than other strategies.

Š 2020, IRJET

|

Impact Factor value: 7.529

|

ISO 9001:2008 Certified Journal

|

Page 3597

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 07 Issue: 08 | Aug 2020

p-ISSN: 2395-0072

www.irjet.net

We accomplished the accompanying: 

DDDS accomplishes exceptionally exact and dependable discovery of laziness.



DDDS offers a non-nosy way to deal with identify languor without the inconvenience and impedance.



Processing makes a decision about the driver's readiness level based on ceaseless eye terminations.



The proposed framework works in both day time and evening time conditions.

All the downsides have been wiped out. In future, this model can be reached out to give alert before dozing by computing the heart beat measure without physical unsettling influence i.e., non nosy technique utilizing changed ECG strategies. Normally in ECG strategy key purposes of body (For model chest, head, wrist and so forth.,) are sticked with wire. In the all-encompassing technique, staying wire might be maintained a strategic distance from. This will lead us to an approach to discover the ideal degree of drowsiness. Further, this model will be reached out to screen the reflect beam from eye utilizing nano camera. On the off chance that the reflection beam is missing, at that point eye is shut in any case eye is opened. We accept that this will make a superior chance to identify languor.

6. CONCLUSION Driver Drowsiness Detection was worked to enable a driver to remain alert while driving so as to decrease vehicle mishaps brought about by tiredness. This paper was worried about lazy drivers and their capability to cause vehicle mishaps. . The driver exhaustion discovery framework ascertains drowsiness level from the driver utilizing a blend of Raspberry Pi 3 Model B and Camera. Raspberry Pi 3 Model B is a processor to compute whether a driver is sluggish. Simultaneously, it recovers pictures from the camera, which is sufficiently quick to identify a driver's highlights continuously. The framework utilizes open source programming called as open CV picture handling libraries, the catches pictures are prepared in this. Raspberry pi and open CV makes the general framework to an ease tiredness location framework.

REFERENCES [1] Hang-Bong Kang, “Various Approaches for Driver and Driving Behavior Monitoring: A Review”, in Proc. IEEE Int. Conf. on Computer Vision Workshops, IEEE 2013. pp. 978-0-7695- 5161-6.

[2] Ruijia Feng, Guangyuan Zhang, and Bo Cheng, “An On-Board System for Detecting Driver Drowsiness Based on Multi-Sensor Data Fusion Using Dempster-Shafer Theory”, in Proc. IEEE Int. Conf. on Networking, Sensing and Control, IEEE 2009. pp. 978-1- 4244-3492-3.

[3]Sahyadehas, K. Sundaraj and M. Murugappan, “Detecting Driver Drowsiness based on Sensors: A Review,” Sensors, 2012.

[4] M. Ingre, T. ÅKerstedt, B. Peters, A. Anund, G. Kecklund, “Subjective sleepiness, simulated driving performance and blink duration: Examining individual differences,” J. Sleep Res., 2006.

[5] Y. Sun, X. Yu, J. Berilla, Z. Liu, and G. Wu., “An in-vehicle physiological signal monitoring system for driver fatigue detection,” in Proc. 3rd International Conf. on Road Safety and Simulation, Indianapolis, USA, Sept. 2011, pp.1- 16 [6] Oraan Khunpisuth, Taweechai Chotchinasri, Varakorn Koschakosai & Narit Hnoohom. ” Driver Drowsiness Detection using EyeCloseness Detection,” in Proc. IEEE Int. Conf. on Signal-Image Technology & Internet-Based Systems, IEEE. 2016, pp. 5090- 5698.

[7] Hua-Zhi Dong, Mei Xie. “Real-Time Driver Fatigue Detection Based on Simplified Landmmarks of AAM,” in proc. IEEE Int. Conf. on Apperceiving Computing and Intelligence Analysis Proceeding, IEEE. 2010, pp. 42448026.

[8] Chen Terrence, Zhou Xiang Sean, ComaniciuDorin,Huang Thomas S. Total Variation Models for Variable Lighting Face Recognition [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(9):15191524

© 2020, IRJET

|

Impact Factor value: 7.529

|

ISO 9001:2008 Certified Journal

|

Page 3598

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 07 Issue: 08 | Aug 2020

p-ISSN: 2395-0072

www.irjet.net

[9] Haycocks G.Driver sleepiness as a factor in car HGV. Accidents, transport research laboratory[J]. Report, 1995, HS, 169.

[10] Home JA,ReynerLA.Sleep related vehicle accidents[J]. BMJ, 1995,30:565-567 [11] NCSDR/NHTSA. Expert panel on driver fatigue & sleepiness. Drowsy driver and automobile crashes[J].

Report, 1998, HS. 808. 707 [12] NHTSA. Expert Panel on Driver Fatigue & Sleepiness. Drowsy driver and automobile crashes Report. 1998 [13] Wang Lei, Mo Yu-Long, Qi Fei-Hu. Automatic Eye Feature Extraction Based on Hough Transform and Eyrlid Deformable Template. J.InfraredMillim[J]. Waves 1999, 18(1): 53-60 [14] Zhao W. Robust Image-Based 3D Face Recognition [D]. Doctoral Thesis, 1999, Center for Auto Research, Univ. of Maryland

Š 2020, IRJET

|

Impact Factor value: 7.529

|

ISO 9001:2008 Certified Journal

|

Page 3599