SURVEY PAPER ON VARIOUS EYE BLINK DETECTION SYSTEMS by IRJET Journal

SURVEY PAPER ON VARIOUS EYE BLINK DETECTION SYSTEMS

Anil C.B1, Ammu Vijayan2 , Anjitha P.S3, Ann Mary George4 , Baselios Paul5

Department of Computer Science and Engineering,Sree Narayana Gurukulam College Of Enginnering,Kadayiruppu,Ernakulam,Kerala,India

Abstract This survey paper aims to provide a compre-hensiveoverviewofthestate-of-the-artineyeblink detection systems, encompassing the methodologies, algorithms,datasets, and applications.Bysynthesizingtheex-istingliterature,we seek to shed light on the challenges and advancements in this field, offering insights into the various approaches employed for accurate and efficient eye blink detection. The provided project have manychal-lenges be- cause the algorithms did notgive correct results. But time to time, several techniques are discovered for eye detection. In this survey paper, we describe and compare many techniques of eye blink detection through table. The purpose ofthisstudy istoprovideathoroughresourcefor scholars, professionals, and hobbyists who are interested in learningabout the changing field landscape. Our aim is to stimulatefurther innovation inthis field and develop a deeper under- standing ofthe importance of eye blink detection throughacriticalstudy ofthematerialthathasalreadybeenpublished.

Keywords: Face Detection, Blink Detection, Eye RegionExtraction,HaarCascade,Imageprocessing Electroocu-lography(EOG)

I. INTRODUCTION

Thehumaneye,withitsintricatemechanismanddynamicmovements,servesasavitalchannelfornonverbalcommunicationandinteraction.Thestudyof eye movements has gained significant attention in variousfields,rangingfrompsychologyandneuroscience to human-computer interaction (HCI) and biometrics.Amongthemyriadaspectsofeyebehav ior,thedetectionofeyeblinksstandsoutasacrucial component due to its potential applications in diversedomains.

Eye blink detection systems have evolved over the years, propelled by advance-

ments in computer vision, image processing, and machinelearningtechniques.Thesesystemsplaya

pivotal role in numerous applications, such as driver monitoring systems, fatigue detection, lie detection, human-computer interaction, and even medicaldiagnostics.

II. RELATED WORKS

[1] The authors Madhumantimaiti et.al (2020) has proposedanInnovativePrototypetoPreventAccidents Using Eye Blink Sensors and Accelerometer ADXL330.Thispapermainlytellsabouttheinnovativeprototypethatutilizeseyeblinksensorsandthe accelerometerADXL330topreventaccidentsrepresents a promising advancement in safety technology. By integrating these sensors, the system aims to enhance real-time monitoring and response mechanisms,particularlyinsituationswheredriver attentiveness is crucial. Monitoring eye blink patternscanprovideinsights into the driver's levelof alertness,enablingthesystemtoissuetimelyalerts or interventions when signs of drowsiness are detected.Thiscouldsignificantlyreducetheriskofaccidentscausedbydriverfatigue,acommonfactorin roadsafetyincidents.The applicationsof thesame is it can be used avaiation,healthcare,industrial safety etc .This Technology is realiable and realworldapplicable.Thedrawbackofthistechnologyis false positives or negatives ,industrial variability,calibrationchangesetc.

[2]AnEyeblinkDetectionsystembasicallyforParalysedpatientshasbeenproposedbyMilanPandey et.al (2020) . The proposed system enables people with severe paralysis to communicate their thoughts and needs. It also helps patient to show their intellectual potential which can sometimes dispose their mental disability diagnosed by the doctor.Theproposedpatientprovidesauniqueand

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

new UI which can be easily controlled by any age grouppatient.Theproposedsystemcombinesexisting techniques in a new way to detect eye motion andeyeblinkdetection.TheEyeMotionAlgorithm isusedtodetermineleft,rightornomotion.TheEye BlinkDetectionAlgorithmisusedtodeterminevoluntaryandInvoluntaryeyeblinkscanbeusedasa blinkgesturesintheproposedsystem.Thesetwoalgorithm helps the patient to navigate efficiently in the proposed system and communicate. The proposed system, Consumer Grade PC/Laptop and Logitechwebcamof$23.53isusedwhichdecrease thecostandboosttheuseofthesysteminvarious environment like private or government hospital personal nursing, home,etc.The system setup and maintenance doesn't require any skilled labor and hencewillreducethecostofthesystem.Themain drawbackissystemaccuracydecreaseinlowlightingconditionsandthewebcamshouldalwaysbein lineofsightwitheyeofthepatient.

[3]AMultimodelHuman-EyeBlinkRecognitionUsingZ-score BasedThresholdingandweighted FeatureshasbeenproposedbytheauthorPuneetSingh Lamba et.al (2021).This paper mainly claims that eye blinks were detected using a blend of 5 weighted features (Vertical Head Positioning, Orientation Factor, Proportional Ratio, Area of Intersection,andUpperEyelidRadius)depictingimperativegen(zscorethreshold),extractedfromthecircles uniquely formed from the eyelids landmarks. Fortestingtheperformanceofthemethod,ZJUeyeblink dataset was used. While implementing the proposed method with the said dataset, it was observedthatwhenthereisanimpulseinOF,itmeans that other features except UER will struggle to reportablinkwithapeak.Incontrast,whenOFhasno impulse signals, it suggests the expected behavior, which peaks in all four features. The multimodal system’sperformancewasincreasedto97.2%(accuracy) with a precision of 97.4%. Other performanceparametersalsoshowedadecentroutine.As afuturescope,morefeaturescanbeincorporatedto increasetheperformanceattributesfurther.

Volume: 11 Issue: 02 | Feb 2024 www.irjet.net p-ISSN: 2395-0072

[4] The authors Taner Danisman et.al (2022) has propsed a paper called Drowsy Driver Detection SystemUsingEyeBlinkPatterns.Theproposedsystemisindependentfromtheheadmovementsasit works within the same frame. Therefore, it has an advantageovertheothersystemsthatusestatistical information from the past frames. In addition, it runsata110fpsrateonIntelXeon2.9GHzCPUfor a 320×240 resolution which is acceptable for realtimescenariosandleavestimeforothertasks.Unfortunately,nocommondatabaseexistsforcomparing our results for drowsiness; therefore we only givetheresultsforeye-blinkdetection.Inaddition, weplannedtoworkontheAv@Cardatabaseforfurthercomparisonofourapproach.Theproposedsystem detects eye blinks with a 94% accuracy and a 1% false positive rate. The author’s experiment showedthattheproposedsystemproducesfastand accurateresultsforthedetectionofdrowsiness.Accordingtotherealworldexperiments,themostimportant factors that affect the performance of our methodarethepresenceofglassesandhighilluminationchanges.Indeed,thepresencesofglassesaffect the core components of the system including facedetection,eyedetectionandsymmetrycalculation.

[5]AnEyeBlinkCountandEyeBlinkDurationAnalysisforDeceptionDetectionhasbeenproposedby theauthorsSabuGeorgeet.al(2021).Inthispaper, analysisofblinkcountandblinkdurationhasbeen donebyconductinga psychological experiment. In the experiment to detect deception, 50 subjects wereaskedasetof10controlquestionsrelatedto name, age, place, house, marital status, entertainment,foodandreligionandtheeyeblinkswererecorded during the interview period using a high speed camera. The questions were repeated for truthandlieresponses.Therecordingwerestarted at the moment of askingthe question, so that only the responses were recorded. The analysis shows thatblinkdurationandblinkcountaremorewhile lying.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

[6]AnEyeBlinkdetectionusingMonocularSystem hasbeenintroduce/proposedbytheauthorsKoichi Takahashiet.al(2020).Inthis paper, theypropose theon-lineeyeblinkdetectionmethod,andtheyapplied the proposed method to the real systems for the purpose of confirming the proposed method workswell.Theirmethodutilizesasimplifiedhead model, and facial feature points can be tracked by particlefiltering.Eyelidmotionsaredetectedasthe deformation from the reference facial image. The proposedmethodrequiresonlyaninexpensiveUSB cameraandrequirednointrusivephysicalmarkers. Fortheexampleofapplications,EEGanalysisforremovingtheeyeblinksignalworkswell,andwecan obtaintheEEGwithouteyeblinknoise.Becauseof the low computing cost, this eye blink detection methodcanworkatover60fps.Hence,itispossible to accurately synchronize the eye blinks and EEG. Additionally, the experimental results indicate the existing of the characteristic eye blink signals, and eyeblinksignaltemplatecanbeobtained.Therefore theproposedmethodwillcontributetorealtimeartifact removal for simple electroencephalograph. Secondly,avatarsystemwitheyeblinkdetectionis introduced. In the proposed system, we can easily puppeteer arbitrary “pmd” 3D model. In order to confirm the effectiveness of the proposed system, theyconfirmthattheproposedsystemisabletoaccurately detect the eyelid motions. According to these results, the effectiveness and availability are demonstratedinthispaper.

[7]AnEyeBlinkdetectionSystem ForVirtual keyboardhasbeenproposedbytheauthorsAfraaZAttiahet.al(2021).InthisPaperThecomputer’scameraisusedtocaptureafacialimage,thentheeyedetection module detects the eye location. Eye blinking is used to select the desired character as the highlightedoneonthevirtualkeyboardlikepressing an “Enter” button. The system is designed for peoplewithadisability.Theresultsshowhighuser satisfactionandprovethebenefitofthesystem.

[8]TheauthorsAkihiroKuwaharaet.al(2021)has proposed a Eye Fatigue Prediction System Using

Volume: 11 Issue: 02 | Feb 2024 www.irjet.net p-ISSN: 2395-0072

BlinkDetectionBased onEyeImage. Inthis Paper, wedemonstratedthatusingOpenCVforfaceimage normalizationcanimprovetheaccuracyofblinkdetectioninimageprocessing.Itmaybeduetotheincrease in face recognition rate by removing informationotherthanthefacialexpressionandreducingthenoisecausedbythefacemovement.Infact, we found our proposed system to be practical for real-worldproductdevelopmentduetoitsfastprocessing speed by using OpenCV and Dlib. On the other hand, if face recognition is not possible, it is still impossible to detect the blink of an eye, so a processing method is needed to compensate for this.

[9]ThepaperAFullyAutomatedUnsupervisedAlgorithmforEye-BlinkDetectioninEEGSignalshas been proposed by the authors Mohit Agarwal et.al (2022).Inthiswork,theyintiallystudiedtheproblem of eye-blink detection in EEG signals. During theirintialtimetheyfindthatregardlesstheabundance of research in this area, the applicability of the proposed algorithms is limited due to one or more requirements of multiple EEG channels, EOG channels, user training phase and manual inspection for the robust detection. In this context, they proposed a fully automated unsupervised algorithm, Blink, to detect eye-blinks in the EEG data. They approach self-learns brainwave profiles for eachspecificuser’seyeblinks,andhencedoesaway with any user training or manual inspection requirements.Blinkcapableoffunctioningonasingle channelEEGaccurately,estimatesthestartandend timestamps of eye-blinks very precisely. They collectedfourdifferentEEGdatasetstoevaluatetherobustnessofalgorithmacrossvariousEEGheadsets, user activities, and eyeblink types, and show that Blinkperformswithanaccuracyofover98%inall casesalongwithanaverageprecisionof0.934.

[10] An Eye Blink Based Biometric Authentication SystemUsinganEvent-BasedNeuromorphicVision Sensor have been proposed by the authors Guang Chen et.al (2021). This work introduces the first-

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

ever neuromorphic sensor based human authenticationsystemusingtheeasy-capturebiometricsof eyeblinks.Oneneuromorphicsensorsuchasadynamicvisionsensorcanbeusedtoaccuratelyidentifyandverifyusers withrelativelysimple computational processes. The detailed results show that thesystemiseffectiveandefficientbasedondifferentfeatureselectionapproachesandauthentication models. The system only uses implicit and highly dynamicfeaturesoftheuser’seyeblinksatamicrosecond level time resolution and at an asynchronous pixel-level vision resolution, which not only ensures security but also avoids recording visible characteristics of a user’s appearance. This work demonstratesanewwaytowardssaferauthentication using neuromorphic vision sensors. In future works, experiments may extend into the effects of the temporal evolution of human growth or other changes. Also, to improve the system’s robustness against attacks, adversarial samples may be added inthefuturetoprovidemoresafetymargin.

TABLE 1. COMPARISON TABLE

PAPER YEAR TECHNIQUES WORKDONE

1. An Innovative Prototype to Prevent Accidents Using Eye Blink SensorsandAccelerometer ADXL330

2.Eyeblink Detection system basically for

2020 Sensors and AccelerometerADXL330

2020 The Eye Motion Algorithm and

The eye blink sensors detect driver drowsiness, while the accelerometer monitors vehicle acceleration

The goal is to establish a connection be-

Volume: 11 Issue: 02 | Feb 2024 www.irjet.net p-ISSN: 2395-0072

Paralysed patients EyeBlinkDetection Algorithm

3. Multimodel Human-Eye Blink Recognition Using ZscoreBased Thresholding and weighted Features

2021 (Vertical Head Positioning, Orientation Factor, Proportional Ratio, AreaofIntersection, and Upper Eyelid Radius) depicting imperative gen (z score threshold), extracted from the circles uniquely formed from the eyelids landmarks

tween specific eye blink characteristics andthelikelihoodofdeception, contributing to the development of more effective deception detection methods

The work involvesdeveloping a multimodal approach forhumaneye blink recognition This includes implementing Z-score based thresholding to enhance accuracy and combining weighted features from different modalities to improve the robustness

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

ofeyeblink recognition

4. Drowsy Driver Detection System Using Eye Blink Patterns

2022 IntelXeon2.9 GHzCPUfora 320×240resolution, Av@Cardatabase

The work involves creating a drowsy driver detection systembyanalyzing eye blink patterns. Researchers develop algorithms to monitorand interpret variations inblinkpatterns, aimingtoreliably identify signs of drowsiness The system utilizes these patternstodetect potential fatigue in drivers, providing timely alertsorinterventions to enhance roadsafety

Volume: 11 Issue: 02 | Feb 2024 www.irjet.net p-ISSN: 2395-0072

Duration Analysisfor Deception Detection

5.EyeBlink Count and Eye Blink

2021 AU45 Detection,Eye

The work involvesdeveloping an

6.EyeBlink detection using MonocularSystem

Blink analysis eye fatigue prediction system usingblinkdetection based on eye images Researchers createalgorithms to analyze eye images,specifically focusing on blink patternstopredict and assess eye fatigue

2020 facial feature point tracking algorithm by using particlefiltering

The work utilizes a simplified headmodel, and facial feature points can be tracked by particle filtering Eyelid motionsaredetected as the deformationfrom the reference facial image The proposed method requires only

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

aninexpensive USB camera and requiredno intrusive and physicalmarkers

7.EyeBlink detection System For Virtualkeyboard

2021 Human Computer Interaction(HCI)

8. Eye Fatigue Prediction System Using BlinkDetection Based on Eye Image

Volume: 11 Issue: 02 | Feb 2024 www.irjet.net p-ISSN: 2395-0072

2021 Image Processing, Dlib, OpenCV, EAR,EARM

The computer’scamera is used tocapturea facialimage, thentheeye detection module detectstheeye location. Eyeblinking is used to select the desired characteras the highlighted one on the virtual keyboard like pressing an “Enter” button.

9.FullyAutomated Unsupervised Algorithm for Eye-Blink Detectionin EEGSignals

2022 eye-blinkdetection in EEG signals, Power SpectrumDensity, Dynamic Time Warping

10. Eye Blink Based Biometric

2021 NeuroBiometric, DAVIS346

Theworkis to detect eye-blinks in the EEG data. Our approach selflearns brainwave profiles for each specific user ’ s eyeblinks, and hence does away with any user trainingormanual inspection requirements Blink capable of functioningona singlechannel EEG accurately,estimates the start and end timestamps of eyeblinks very precisely

The work introduces the first-

The work demonstratedthat using OpenCV for face image normalizationcanimprove the accuracy of blink detection in image processing

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

AuthenticationSystem

Using an EventBased Neuromorphic Vision Sensor ever neuromorphic sensor based human authentication system usingtheeasy capturebiometrics of eye blinks. One neuromorphic sensorsuch as a dynamic vision sensor can be used to accurately identifyandverify users with relatively simple computational processes

TABLE 2. ADVANTAGES AND DISADVANTAGES OF RELATED WORKS

PAPER ADVATAGES DISADVANTAGES

1. An InnovativePrototype toPreventAccidents Using EyeBlinkSenEarlyAccident Prevention, Real -time Monitoring.CustomizableSensitiv-

False Positives and Negatives, Privacy Concerns,Technical Challenges,Cost and Accessi-

Volume: 11 Issue: 02 | Feb 2024 www.irjet.net p-ISSN: 2395-0072

sors and Accelerometer ADXL330 ity, Integration with Vehicle Systems, Data logging andAnalysis biltiy ,User acceptance, Maintenance

2. Eye blink Detection system basically for Paralysed patients Communication Aids, Indepedence, CostEffective, Non Invasive ,Real - time Monitoring Limited Communication Complexity,Accuracy Issues, Integration Challenges, Privacy Concerns, Fatigue due to continuous blink

3.Multimodel Human-Eye Blink Recognition Using Z-scoreBased Thresholding and weighted Features Accuracy, Robustness, Adaptability, Integration, Feature Importance Complexity, Computational Overhead, Data Dependency, Limited Generalization

4. Drowsy Driver Detection System Using Eye BlinkPatterns EarlyWarning System, Realtime Monitoring, ReducedAccidentrisk False Positive, Individual Varibility, User Acceptance, Technological Limitations, cost, Ethical Considerations

5. Eye Blink CountandEye Blink Duration Analysis for Deception Detection Automated Data Collection, Realtime Monitoring, Objective Measures Dual signaling, False positive and negatives, Privacy Concerns

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

6. Eye Blink detection using Monocular System

Volume: 11 Issue: 02 | Feb 2024 www.irjet.net p-ISSN: 2395-0072

Non-intrusive, Cost effective, Simplicity, Real-time Monitoring

7. Eye Blink detection System For Virtualkeyboard

Handsfreeoperation, Gesture Precision, Enhanced Security, Versatility, Natural Interaction

8.EyeFatigue Prediction System Using Blink DetectionBasedon EyeImage Early Fatigue Prediction, User Health Improvement, Customizable Alerts, Productive

9. Fully Automated Unsupervised Algorithm for Eye-Blink DetectioninEEG Signals Efficiency, Scalability, Consistency, Real-time Monitoring

10. Eye Blink

Based Biometric Authentication System Using an EventBased Neuromorphic VisionSensor

Highly Personal, Robustness to Lighting Conditions, Realtime Processing

Ambiguity in gaze tracking Limited Depth Perception ,Challenges with glasses andeyemakeup

False positive and negatives, fatigues issues, Limited Input Options, PrivacyConcerns

False Positives, Accuracyissues, Privacy Concerns

Accuracy, False positives,Complexity, Lack of adaptibility

FUTURE SCOPE

Eyeblinkdetectionsystemswillcomeintouseinhuman-computer interaction, wearable technology, marketing,healthcare,emotionrecognition,privacy, learning, gaming, and sports performance evaluation in the future. It is expected that these innovations will improve graphical user interfaces, improve safety, monitor health conditions, offer biometric verification, and make significant improvementsto variousindustries throughtheanalysisof eye blink patterns for beneficial data and applications

CONCLUSION

Theconclusionofthispaper,weunderstandthedifferent methods to implement eye blink detection system.AndLiteraturereviewofobjectinrealtime environment and non real time environment using manydifferentapproaches.Someofthepaperseems quitegoodlikepaper[7],tomakeitmoreadvanced wecanusepredictivespeechrecognition.Mostofthe literature survey papers are having privacy issues thatcanberemovedbyfuturetechniquestoo.The mainaimofthispaperidentifyanobjectapproxand usingdifferentmethodsandmanyapproachestoget finalresult.

REFERENCES

Privacy Concerns,Cost,Vulnerability, Sensitive to Eye healthchanges

IEEE

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

[3] Puneet Singh Lamba, Deepali Virmani, Manu S. Pillai , Gopal Chaudhary, “Multimodal Human EyeBlinkRecognitionUsingZscoreBasedThresholding andWeightedFeatures”,InternationalJournalofInteractiveMulTimediaandArtificialIntelligence,Vol. 7,Nº42021

[4]TanerDanisman,IoanMariusBilasco,Chaabane Djeraba,NacimIhaddadene,“DrowsyDriverDetectionSystemUsingEyeBlinkPatterns”,International Conference on Machine and Web Intelligence (ICMWI 2010), Oct 2010, Alger, Algeria. pp.230-233, ff10.1109/ICMWI.2010.5648121ff.ffhal00812315

[6] Koichi Takahashi and Yasue Mitsukura, “Eye BlinkDetectionUsingMonocularSystemanditsApplications”, 2012 IEEE RO-MAN: The 21st IEEE InternationalSymposiumonRobotandHumanInteractiveCommunication.September9-13,2012.Paris, France.

[8] Akihiro Kuwahara ,Kenichi Nakashi ,Rin Hirakawa,YoshihisaNakatoh,HidekiKawano,“Eye FatiguePredictionSystemUsingEyeBlinkDetection BasedOnEyeImage”,2021IEEEInternationalConference on Consumer Electronics (ICCE) | 978-17281-9766- 1/20/$31.00 ©2021 IEEE | DOI: 10.1109/ICCE50685.2021.942768

[9]MohitAgarwal,RaghupathySivakumar,“Blink:A Fully Automated Unsupervised Algorithm for EyeBlink Detection in EEGSignals”, 2019 57th Annual Allerton Conference on Communication, Control,

Volume: 11 Issue: 02 | Feb 2024 www.irjet.net p-ISSN: 2395-0072

andComputing(Allerton)AllertonParkandRetreat CenterMonticello,IL,USA,September24-27,2019

[10]GuangChen,Member,IEEE,FaWang,Xiaoding Yuan,ZhijunLi,SeniorMember,IEEE,ZichenLiang, and Alois Knoll, Senior Member, IEEE, “NeuroBiometric: An Eye Blink Based Biometric AuthenticationSystemUsinganEvent-BasedNeuromorphicVisionSensor”,IEEE/CAAJOURNALOFAUTOMATICA