Theprimaryfocusoftheproposedmodelis toidentifywhetherapersoniswearingamaskornot,basedonthevisualsof image/videostream.
International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056 Volume: 09 Issue: 01 | Jan 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page1324 Deep Learning Assisted Tool for Face Mask Detection Karthik Prasad1, Madhava Raj B2, Gokulnath S3, Sanjan Govind K4
1. INTRODUCTION
2. LITERATURE SURVEY
Therearemethodsintheliteraturethatareappliedonthefacemaskdetectionmodel.[4]combiningimagesuper resolution andclassificationnetwork(SRCNet)developedamethodforidentificationfacemask wearingcondition,whichdetermined threecategoriesofclassificationproblemonthebasisofunconstrained2 Dimensionalfacialimages.[5]thispaperproposesa differentapproachforfacerecognitionwithdeepneuralnetwork.Inthismethod,extractedfeaturesoffacearegivenasinput tothedeepneuralnetworkinsteadofrawpixeldata.ThishelpsinbetterperformanceonYaleFacedatasetbyreducingthe complexityandimprovedaccuracy.[6]inthispaper,usingdeeplearningandimageprocessingtechniques,proposesamethod to detect human faces and classify them. This model uses MobileNetV2 [7] architecture and for optimization ADAM [8] optimizerisused.Proposedmodelisdiscussedinsection III,Methodologyinsection IV,Resultsinsection Vandfinallypaper isconcludedinsection VI
3. PROPOSED MODEL
Coronavirus(COVID 19)isaninfectiousillnesscaused bytheSARS CoV 2virus.Thefirstcasewasinitiallydiscoveredin Wuhan(China)bytheendof2019,laterspreadingthroughouttheglobalbecamepandemic.Asofnow(3rdJanuary2022)5.4 millionpeoplehavesuccumbedtoCovid 19[1].Currentlytheworldisinthedangerofanotherwaveofvirus.Inspiteofhigh vaccinationrates,newcasesareinsurgeacrossEurope.WHOhasrecentlywarned,ifstrictrestrictionswerenotimposedas soonaspossible,maycause0.5millionsdeathsbytheendofFebruary[2].Italsoissuedseveralguidelinestocheckthefaster spreadingofthevirussuchaswearingfacemask,maintainingsocialdistance,regularhealthcheckups,vaccinationetc[3].In thisprojectwehaveworkedoverasystem,whichcaneasilyencounterwhetherthepersoniswearingfacemaskornotinthe publicplaceslikecinemahalls,shopsetcwheretheCCTVisfunctional.Thesystemcanhelptopreventtherapidtransmission of the viruses. Face recognition is one of the most promising field of computer vision. Recognizing the face and verifying whetherthepersoniswearingafacemaskornotautomaticallyisthekeyprincipleofourproject.
Abstract - COVID 19 pandemic has already caused the huge havoc around the world with its recurring multiple waves. Many have lost their loved ones. Great number of people lost their jobs. World economy is still struggling to recover the losses caused by the pandemic. The pandemic has made a significant riffles in the lives of people around the globe. The World Health Organisation(WHO) has issued the several preventive measures to avoid the transmission of virus like measure masks in public gathering, maintain social distance, vaccination etc [3]. Checking whether the public is following the safety guidelines is a tedious job for the authorities concerned. Our team has worked over a model, which can easily identify whether a person is wearing mask or not in public gatherings like cinema theatres, malls, govt offices etc. where the CCTV is functional. The system can prevent the transmission of virus. This model has been developed using the various machine learning frameworks like Tensorflow and Keras. We have used MobileNetV2 architecture in this model.
Keywords__ Tensorflow, Covid 19, MobileNetV2, Deep Learning, Face Mask, CNN
1,2,3,4Student, Electronics and Communication Engineering, Vivekananda College of Engineering and Technology, Puttur, Karnataka, India ***
disk Load face mask classifier
Todetectwhetherapersoniswearingornot,TensorFlowandKerasframeworksareused.Hereinthiscurrentproposed system, initially training of the model is done using the datasets which are collected from Kaggle. For training purpose TensorFlowandKerasareusedtogether.Oncethetrainingofthemodeliscompleteweloadthetrainedmodel,herebyreal timevideostreamfacesareclassifiedaccordingly.InthisproposedsystemweusedMobileNetV2architecturesinceitisvery muchhelpfulintrainingthemodelwithhugedatasetandmaintainingthequalityofimageswhichweusedtotrain.After loadingofimages,sincewecannotdirectlytrainedmodelusingrawimages,theyareconvertedintoarrays,.Afterthatusing MobileNetimagesareprocessedandappendedintothedatalist.Themostsignificantpartofthiscurrentsystem areface recognitionandmaskdetection.MobileNetV2andOpenCVareexplicitlyusedforthispurpose.OncetheROIi.e.facehasbeen detectedbythesystem,aredorgreenrectangularboxwillbeplacedsurroundingtheface,dependingonfacewithorwithout mask.Around1800+imagesofpeoplewithandwithoutmasksarecollectedfromKaggleforthispurpose.Byusingthese imageswetrainmodelwhichwillclassifiesin2categoriesnamely,“withmask”and“withoutmask”.Whenapersonappearsin frontofthecamera,initiallythesystemwillrecognizethefaceandlateritwilldetectwhetherapersoniswearingmaskornot. It will also display the correctness of the weared mask with rectangular box around the face. The monitoring will done continuouslywiththehelpofvideostreamandrealtimedataofanalysiscanbeobtained.Ifthepersonisfoundwithoutmask, mask mask to from Detect faces in steamimage/video face ROI ROI
Fig. 1. Block diagram of the proposed model
ApplytrainedfacemaskdetectormodeltoeachfaceROI(RegionofInterest)todetermine“mask”or“nomask”afterthisweget theresultwhetherthepersoniswearingthemaskornot.
classifier Keras/Tensorwithflow load face mask classifier
Extract each
Show result Apply face mask classifier to each face
disk
International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056 Volume: 09 Issue: 01 | Jan 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page1325 #Phase1 #Phase2
determineto “mask” or “no mask”
Load face
Infirstphase,wewillgatherthewithandwithoutthemaskimages.Thisdatasetconsistsof1800+imagesbelongingeitherof thetwoclasses.Insecondphase,wewilltraintheclassifierforfacemaskdetection,andinthefinalphase,wewillusethe trainedmodeltodistinguisheachfaceaswithmaskorwithoutamask.
detector Train face
Inthesecondphase,weloadfacemaskclassifierfromthedisk.Wehavetoadddatasettodetecttheface.Thewehavetoturn Onthecameratodetectthefacewhethertheyarewearingthemaskornot.
Fig. 3. Image dataset (without mask)
STEP1:Pre processingofData:
Methodologyofproposedmodelwillbediscussedindetailinthissection.
Fig. 2. Image dataset (with mask)
4.1 Steps involved in face mask detection:
a)DataPre processing:InDatapreprocessingstep,datawillbeprocessedintodesiredformat.Itinvolvestransformationof datafromagivenformattoasrequiredformatbythemodel.Thisprocesseddatawellfitintothemodelandwillbecompatible withdifferententities.WiththehelpofNumpyandOpenCVlibrariestheproposedmodelperfomsoperationsonimageand video.
hispicturewillimmediatelysenttotheconcernedauthoritysothattheycanfurtheraction.Sinceworldisinthe vergeof anotherwaveofcoronavirus,theproposedmodelcanbeconsidereddirenecessity.
International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056 Volume: 09 Issue: 01 | Jan 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page1326
4. METHODOLOGY
Dataset:Ourproposedmodelusesdatasetcontainingcroppedimagesoffacewith variablerotationanddifferentorientation. This makes our data more diverse. Images were collected from images dataset offered by Kaggle, a open source online communityofdatascientistsandmachinelearningpractitionersandfewwithinthepublicdomain.Thedatasetfoldercontains twosubfoldersnamelyWithmaskandWithoutmaskeachcontainingtheimagesofrespectivetypes.Eachsubfoldercontains 3600+images.Accordinglytheyarelabelledandtrainedinourmodel.WeuseMobileNetandOpenCVforautomatedrealtime maskdetection.
International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056 Volume: 09 Issue: 01 | Jan 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008
Certified Journal | Page1327
d)ImageReshaping:MajorityoftheCNNsrequirestheiriputstobefined tuned.Sizeofinputimagesmustbeidenticaltothe the 3D feature tensor. It is required to reconfigure the inputs before we provided them as inputs to the model. They are compressedtothepixelrange[0,1].Afterthattheyaretransformedto4D arrays.LastNNwillgivetwopossibleoutputsi.e. wheretheinputimageismaskedorunmaskedintermsofbinaryvalues.
STEP3:Model BuildingandTraining Inourproposedmodel,weloadthepre processeddatasetandtrainthemodelusingalgorithmonlabelleddataset.Here, loadedimagesareresizedandaretransformedintonumpybasedarray. Inourproposedmodel,thedatasetisloadedandfor thepurposeoftraininglabelledimagesareused. Atthisstage,imagesfromthedatasetareresizedandconveredintonumpy basedarray.Inthismodel,weuseMobileNettoconstructthelightweightdeepCNNs.UsingTensorFlowmodelistrained. MobileNetarchitectureisusedherewhichcanbeconsideredasthemainstayandtensorflowframeworkisusedtotrainthe model.Wekeepinitiallearningrate,INIT_LR=1e 4,EPOCHS=20, batchsizeBS=32. Inthismodel,forthepurposeof face detectionwebcamisused.Ifthemodelrecognizestheface,itwillplaceredsquareboxandaccuracyaroundtheface.
Herewesplitthedatasetintotrainingsetandtestingtest.Inthetrainingset,therewillbeimagesthatweusedtotrainthe modelandusingthetestingsetweevaluateourCNNmodel.Inoutmodelwesettestsize=0.2,whichwillsplit80%and20% datatotrainingsetandtestingsetrespectively.
Fig. 4. RGB to Gray Scale conversion of a image of size 100x100
a)Buildingthemodel:CNNisplayingasignificantroleinmoderndaycomputervisionapplications.WeusedSequentialCNNin our model. Sequential CNN is used in the current method. Output of the first convolution layer is fed into the activation function calledRectifiedLinearUnit(ReLU)[9]followedbytheMaxPoolinglayer.Theinputsthatarefedintofirstconvolution layersisconvolvedwith180filtersofkernelsize3X3.Alongwithfirstweneedtoprovidetheinformationofshapeofthe expected input because the model must have the knowledge of shape of the input will be receiving. Then we use “relu” activationfunctinforConv2Dclass.FortheConv2Dlayer,ReLUactivationfunctionisused.Thisoptimizationparameterhasall thepropertiesoflinearmodelsthatcaneasilybeoptimizedwithdifferentgradient descenttypes.Itperformswellcomparedto othertypesofactivationfunctions.ThenMaxPoolinglayerisusedtodecreasethedimensionsofresultfromthepreceding operation.theoutputofpreviousoperationisreducedbyMaxPoolinglayer.HereMaxPoolinglayerofPoolsize7x7isused. TheresultingoutputofpoolinglayerisagainconvolvedwithConvolutionlayerwithfiltersizeof128 andkernelsizeof3X3. TheresultingoutputisagainfedintoReLUactivationfunctionfollowedbyMaxPoolinglayer.Thisresultingmatrixoffeatures isflattenedusingFlattenlayerandthisflattenedmatrixfeaturesintheformvectorisfedasinputparameterstothefully connected neural network. In order to reduced the overfitting a Dropout layer is used. In the final dense layer, softmax activationfunctionisusedwhichclassifiesthefaceintheinputimageas‘with’maskor‘without’mask.
STEP2:SplittingtheDataset
b)DataVisualization:Itcanbedefinedastheprocessoftransformingabstractdatastoeasilyinterpretablerepresentations.It gives the user real time trends, new insights about the data. It is useful in determining pattern in a given dataset. In the proposedmodel,wefirstinitializetwoemptylistsnamely‘data’and‘labels’whichstorethearrayrepresentationsofimages anditscorrespondinglabeli.e.whethertheimageismaskedornon maskedimage.Twosub foldersinthedatasetfolderis iteratedbyaforloop,eachsub folderatatime.Initiallytheimagesarelabelledassameasthatofthenameofthefolderthey arepresent.Afterlabellingthedatasetthroughforloop,the‘labels’listintheformof[‘withmask’,‘withoutmask’,….]matching theimages.Sincewecannottrainthemodelwithcategoricaldata,itneedstobeencodedas0’sand1’s.WeperformOne hot encodingonthe‘labels’inordertotransformitintobinarydatausingthecommandbelow.
c)RGBtoGrayscaleconversion:RGBimagescontainsmanyfeaturesthoseweremaynotbeusefulforthetrainingalgorithm andtheytakelongtimetoprocess.Inordertoextractthefeaturesthatwereunique,weneedto the imagefromRGBto grayscale Itwillreducedthecomplexityofthecode,difficultyofvisualization.Itwillincreasethespeedoftrainingthemodel.
Fig. 5. Convolutional Neural Network architecture
STEP5:Importingthetrainedmodel: ByusingourLaptop’swebcam,real timefacemaskisdetected.Priortothat,wehavetoimplementthedacedetectionmodelin ourprogram,usedforreal timeinput.
Fig. 6. Training Model accuracy and loss curves
b)Trainingthemodel:Onaccordingtothecompilationmethod,thelearningmethodologyneedstobeconfiguredaccordingto the requirement. Optimizer used is “adam”, which uses Adam algorithm, a stochastic gradient descent method.. Adam is consideredtobethebestamongadaptiveoptimizers.Categoricalcrossentropyisusedaslossfunctionhere.Wesetthemetrics to“accuracy”sinceproblemisaclassificationproblem.ForbetteraccuracyMobileNetV2canbeused.
STEP4:LabelingtheInformation: Oncethemodelisbuilt,resultswill belabelledwithbinarylabelsasperthebinaryprobabilities,i.e.[“0”for“withoutmask”, “1”for“withmask”].UsingRGBvaluescoloroftherectanglethatboundsalsobechanged.[“GREEN”for“withmask”,“RED”for “withoutmask”].
International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056 Volume: 09 Issue: 01 | Jan 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page1328
STEP6:DetectionofFaceswithandwithoutMask: Inthisfinalstep,usingtheCVlibrary,webcamwillopenforinfinitelooptillwemanuallyexittheloop.Herefacialrecognition willdonebycascadeclassifier.Thecodewebcam=cv2.videocapture(0)isusedtoturnontheprimarycameraofthePC.Wecan usemultiplecamerasbyvaryingtheargumenttotheabovecode.Themodelwillestimatetheprobabilityofpossibilityofeach class.Classwithhigherprobabilitywillbechoosenandrespectivetagswillbeplacedaroundtheface.
5. RESULTS AND DISCUSSIONS
International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056 Volume: 09 Issue: 01 | Jan 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page1329 Fig. 7. Performance Metrics Histogram graph Byintegratingeverycomponentofourmodel,wegettheaccuratemaskdetectionsystem.MobileNetV2classifierisutilizedin the current model. The developed model is capable of detecting the facemask in videostream even with many faces and differentorientation. 5.1 Face mask detect from real time Videostream:Fig.8.Output when no mask Fig. 9. Output when mask weared improperly
International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056 Volume: 09 Issue: 01 | Jan 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page1330 Fig. 10. Output when Mask weared properly Table 1. Comparison of current model with other models Author Method Classifier Result(Accurecy) 1.Ejazetal.[10] Viola Jonesalgorithm Principle component ananlysis(PCA) Nearest neighbor (NN) classifier Maskedface72% Non maskedface95%. 2.ArjyaDasetal.[11] Sequential Convolutional NeuralNetwork cascadeclassifierandCNN 95.77% 3.Parketal[12] RecursivePCA PCA 90% 4.Nieto Rodr´ıguezetal[13] Viola Jonesfacedetector Cascadeclassifier FalseRecal>95%positive<5 Table 2. Performance Metrics of current Face Mask Classifier: precision recall f1 score support With_mask 0.99 1.00 0.99 383 Without_mask 1.00 0.99 0.99 384 accuracy 0.99 767 Macroavg 0.99 0.99 0.99 767 Weightedavg 0.99 0.99 0.99 767 6. CONCLUSION The current model automaticallydetects whether a person appearing in front of the camera is wearing face mask or not without human intervention.Resultsof processcan be used bytheconcernedauthoritiesto takelegal actionagainstthe culprits.ThisproposedmodelusesComputerVisionandmachinelearningalgorithmstoensurepublicsafetyagainstCOVID 19 virus. TheproposedfacemaskdetectionmodelistrainedusingMobileNetV2architecturewiththehelpofPythonlibraries suchasOpenCV, TensorFlow, Keras.The proposedmodel can beimplemented in variousplaceslike commercial offices, schools colleges,airports,railwaystations,shoppingmalls,etc.Goodaccuracyhasbeenachievedandthereisgreaterscopefor realtimeimplementationandoptimization. REFERENCES [1]WorldHealthOrganization(2022),WHOCoronavirus(COVID 19)Dashboard, Available: https://covid19.who.int/[Accessed03.01.2022]
Rodríguez,Adrian,ManuelMucientesandVíctorManuelBrea.“SystemforMedicalMaskDetectionintheOperating RoomThroughFacialAttributes.” IbPRIA (2015).
[9]Nair,Vinod,andGeoffreyE.Hinton."Rectifiedlinearunitsimproverestrictedboltzmannmachines."InIcml.2010.
[2]INSIDENE,“WHOsaysEurope,CentralAsiacouldsee5lakhsCOVIDdeathsbyFeb2022”, Available: https://www.insidene.com/who says europe central asia could see 5 lakhs covid deaths by feb 2022/amp/ [Accessed: 03.01.2022]
[4]Qin,Bosheng,andDongxiaoLi."Identifyingfacemask wearingconditionusingimagesuper resolutionwithclassification networktopreventCOVID 19."Sensors20,no.18(2020):5236.
International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056 Volume: 09 Issue: 01 | Jan 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page1331
[8]2018.Kingma,DiederikP.,andJimmyBa."Adam:Amethodforstochasticoptimization."arXivpreprintarXiv:1412.6980(2014).
[5]Priya Gupta, Nidhi Saxena, Meetika Sharma, and Jagriti Tripathi. "Deep neural network for human face recognition."InternationalJournalofEngineeringandManufacturing(IJEM)8,no.1(2018):63 71. [6] Bhadani, Amrit & Sinha, Anurag. (2020). “A FACEMASK DETECTOR USING MACHINE LEARNING and PROCESSING TECHNIQUE”13. [7]Sandler,Mark,AndrewHoward,MenglongZhu,AndreyZhmoginov,andLiang ChiehChen."Mobilenetv2:Invertedresiduals andlinearbottlenecks."InProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition,pp.4510 4520.
[10] M.S.Ejaz,M.R.Islam,M.Sifatullah,A.Sarker,“Implementationofprincipalcomponentanalysisonmaskedandnon maskedfacerecognition”, 20191stInternationalConferenceonAdvancesinScience,EngineeringandRoboticsTechnology (ICASERT)(2019).
[3]WHO,“Adviceforthepublic:Coronavirusdisease(COVID 19)”, Available: https://www.who.int/emergencies/diseases/novel coronavirus 2019/advice for public/[Accessed:03.01.2022]
[11]A.Das,M.WasifAnsariandR.Basak,"Covid 19FaceMaskDetectionUsingTensorFlow,KerasandOpenCV,"2020IEEE 17thIndiaCouncilInternationalConference(INDICON),2020,pp.1 5,doi:10.1109/INDICON49873.2020.9342585.
[12]Jeong SeonPark,YouHwaOh,SangChulAhnandSeong WhanLee,"Glassesremovalfromfacialimageusingrecursive errorcompensation,"inIEEETransactionsonPatternAnalysisandMachineIntelligence,vol.27,no.5,pp.805 811,May2005, doi: [13]10.1109/TPAMI.2005.103.Nieto