Smart Attendance System using Face-Recognition

1, 2Student, Computer Engineering, Ramrao Aidik Institute of Technology(India)

3Student, Electronics and Telecommunication Engineering, Fr. C. Rodrigues Institute of Technology(India) ***

Key Words: face recognition, image processing, face detection,SiameseNetworks

Volume: 09 Issue: 03 | Mar 2022 www.irjet.net

1.INTRODUCTION

p ISSN: 2395 0072

Abstract Everyinstitute,college,andorganization,largeor little, has an attendance marking system. We’ve built a methodthatusesthelattertosimplifytheformer,thanksto huge advances in the field of image processing. Face Recognitionisbecomingmorepopularthanotherbiometric verificationmethodsduetoitssimplicity,non invasiveness, andlackoftouch.Thesystem’smajorgoalistoidentifyand recognizefacesinareal timeenvironment,matchthemwith data in the database, and record their attendance. This is intended to make the time consuming manual attendance process more efficient. This also overcomes the issue of authenticationandproxiesbecausebiometricsareone of a kind,andfacialtraitsusedforFaceRecognitionareoneof them. For face detection and recognition, the designed system uses OpenCV, dlib, Face Recognition libraries, and One ShotLearning,whichtakesjustoneimageperpersonin thedatabaseandsosavesspacewhencomparedtostandard training testingmodels.

Manualattendancesystemsaretraditionalsystemsinwhich a teacherorlecturertakesstudents’attendancebycalling names or signing an attendance sheet. Such attendance systems rely entirely on students acting in a fair and consistent manner. Although it is a low cost system, it is extremely vulnerable to human error or manipulation. A studentmaybemistakenlymarkedpresentbytheteacherif another student answers it on a roll call, or a student can forgesignaturesonthesheet,resultingin’proxyattendance’ [16].

Thereisaninherentpositiverelationshipbetweenstudents’ attendance in schools and colleges and their academic performance, according to research [1]. And, in order to maintainthisrelationship,itisnecessarytoencouragetheir presence and performance in the classrooms, so that studentsaremotivatedtokeepupwiththeprogressofthe subjects being taught in class, thereby increasing their participation in school/college. Attendance management systems have been implemented in schools, colleges, and universitiesall overthe worldusinga varietyofmethods. Despitetheirhighusability,thepracticalityofthesesystems is a little questionable. The face recognition based attendance system is one such system that has recently gained traction. Face recognition is a technique for identifying,verifying,ordistinguishingasubjectbasedonan imageorvideoofthesubject’sface.Itemploysabiometric identification method that uses facial and head measurementstoverifyaperson’sidentity.Facerecognition biometric systems use computer algorithms to pick out specific,distinguishing featuresofa person’s face,such as thespacebetweentheeyesortheshapeoftheface.These characteristics are converted into a mathematical representation,suchasanarrayormatrix,andcomparedto the characteristics of other faces in a face recognition database.Afaceencodingisdataaboutaspecificfacethat differs from a photograph in that it is designed to only

includecertain detailsthat canbeusedtodistinguish one face from another. This system requires any device with digitalphotographictechnology,suchasawebcamoraCCTV camera,togenerateandobtaintheimagesanddataneeded to create and record the biometric facial pattern [11]. Variousfacialrecognitionalgorithmshavebeendeveloped over the years to recognize people regardless of their environment,lighting,angle,orfacialexpression.Basedon itsperformanceinothersecurityapplications,itappearsto be a promising approach for student attendance systems that can help solve problems associated with current systems. A system that uses facial recognition to assess students’attendanceusingmachinelearningalgorithmsis proposed in the proposed paper. The One Shot Learning approach is used in the proposed system, which requires onlyoneimageofeachstudenttotrainthesystemthatwill beusedtodetecttheirfaces,generatetheirfaceencodings, andmarktheirattendance.Theattendancewillberecorded onanExcelsheet,whichstaffandfacultymemberswillbe able to assess and evaluate. We used a pre trained deep neural network called face recognition, which was built usingdlibandhasanaccuracyofnearly99.38percentonthe LFW dataset [12] to implement the OneShot Learning approach.Totestthesystem’sstabilityandrobustness,we testeditonafewstudentsfromourcollegeinvariouslight settings,camerasettings,andocclusions.

Mr. Rajvardhan Shendge1, Mr. Aditya Patil2, Mrs. Tejashree Shendge3

Varioussystemsarecurrentlyinusetomanageandassess student attendance at universities. Even though these systems are extremely usable, their practicality and constraints pose a problem in the process, as previously stated.Thefollowingareafewofthesystemsinplace:

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

2 1 Manual attendance system

2. LITERATURE SURVEY

that combines Eigenface, Principal Component Analysis (PCA), and Linear Discriminant Analysis (LDA). The facial featuresobtainedthroughthesealgorithmscanthenbeused toidentifystudentsand,asaresult,marktheirattendance. Authors Kiran et al. [6] developed a face recognition attendance system that employs Eigenface, Haar Cascade Classifier, and Principal Component Analysis (PCA) algorithms. Their method was to take real time images of students,comparetheextractedEigenvaluestothoseinthe database, and mark attendance based on the recognition resultfromPCAanalysis.Thissystemhada97%accuracy rate when tested on a database with images from 70 students.TheattendancesystemproposedbyD’Souzaetal. [4] is based on the Haar Cascade Classifier and the Local BinaryPatternHistogram(LBPH)algorithm.Theirproposed system would take group photos of students during class hours,performfacialsegmentationandidentification,and update attendance accordingly. Harikrishnan et al. [5] created an attendance system using the Haar Cascade Classifier and the Local Binary Pattern Histogram (LBPH) algorithm in another implementation of a similar system. Theirsystemachievedamaximumaccuracyof74%when used in various environments such as lighting and occlusions.

3.

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

3.1METHODOLOGYOneShotLearning Model

2.2 RFID Based attendance system

The One Shot Learning Model is the foundation of our system.It’saclassificationtaskinwhichonesampleisused to classify a large number of future samples. Face recognitionsystemsbasedontheOne ShotLearningModel learnarichlow dimensionalfeaturerepresentationknown asafaceencoding,whichcanbeeasilycalculatedforfaces andcomparedforverificationandidentificationtasks[14]. Consider the case of a face recognition system for a timekeepingsystem.Imagesofmultiplefacesmakeupthe inputdataset.ConvolutionalNeuralNetworks(CNN) trained models require a large number of images to train and achievehighaccuracy.Ifthereareafewminorchangesin the dataset, these models must be trained iteratively. If a studentdropsoutofcollege,thedatasetmustbeupdatedby deleting the student’s images, and the model must be retrained. In addition, when a new student is admitted to college,theirimagesmustbecollected,andthemodelmust beretrained.Thisprocedureconsumesasignificantamount of time and manpower. To address this, the One Shot Learningmodelcanbeused,whichtakes alotlesstimeto trainbecauseonlyoneimageofthestudentisrequired.The SiameseNetworkiswidelyusedtoimplementtheOne Shot LearningModel.Figure1showshowtheSiameseNetwork performs facial recognition. A similarity function is the foundationofany Siamese Network.ASiameseNetwork’s architectureconsistsoftwoparallelnetworks,eachtakinga differentinput,andcombiningtheiroutputstogenerate a prediction. A Siamese Network for face recognition is a neural network that learns a function f(d) that takes two

p ISSN: 2395 0072

Bluetooth based systems collect information about the students present in the class and record their attendance using Bluetooth signals from their phones. This system appears to be very practical, as nearly 95% of college studentshavetheirphoneswiththem.Tomakethesystem perfect, it can also implement proxy removal methods. However, the system’s main flaw is its lack of usability. A Bluetooth baseddevicecanonlyconnectto8otherdevices atatime.ThisisduetotheMasterandSlaveconcept,which limitsadevice’sconnectiontoonlyeightotherdevicesata time. As a result, this system can only be used when the numberofstudentsinaclassroomisinthesingledigits[18]. After more thought, it was discovered that every existing attendancemanagementsystemhadflawsthattaintedthe process. Problems caused by ’proxy attendances’ will be eliminated by using facial detection and recognition as a parameterofattendancegeneration,asonlythosestudents presentinthelecturewillbemarkedpresent.Becauseevery classroomhasalaptopandawebcam,thecomponentsare alsoinexpensive.Themainstrategyistocomparetheface encodings of the image captured in real time with those alreadystored in thedatabase, whichcanthen be used to markattendanceifamatchisfound.AReal TimeMultiple Face Recognition using Deep Learning on Embedded GPU SystemwasproposedinthepaperbyauthorSaypadithetal. [9].Facedetectionandtrackingwereimplementedusinga Convolutional Neural Network (CNN). Author Deeba et al. [10] used a similar approach to develop a Local Binary PatternHistogram(LBPH) basedEnhancedReal TimeFace Recognition system that can recognize faces in low and highlevel images in real time. A model of an automated attendancesystemwasproposedbyauthorsAkbaretal.[7]. Theirsystemdetectsandcountsstudentsastheyenterand exittheclassroomusingacombinationofRadioFrequency Identification(RFID)andFaceRecognition.Itkeepstrackof each student’s attendance records and provides pertinent informationasneeded.AuthorSmithaetal.[8]usedHaar Cascade Classifier and Local Binary Pattern Histogram (LBPH)forfacedetectionandrecognitionintheirautomated attendancesystem.Faceswerecapturedusingalivestream video of the class in their system, and attendance was recorded, which could be accessed as a CSV file. For face recognition,authorSawhneyetal.[3]useahybridalgorithm

RfidistheabbreviationforRadioFrequencyIdentification. StudentsaregivenRFIDcards,whicharescannedusingan RFID reader to mark their attendance at universities. The RFIDsystem’smainflawisitslackofpracticality.RFIDtag cardsareprohibitivelyexpensive,andpurchasingthemfor anentireuniversityisnotfeasible.Anotherdisadvantageis thatifstudentsarenotsupervised,theycanscanmultiple RFIDcardsonthereader,resultinginproxyattendance.If notsupervisedorcaredforproperly,theRFIDreaderisalso susceptibletodamage[17].

Volume: 09 Issue: 03 | Mar 2022 www.irjet.net

2.3 Bluetooth based attendance system

2.)Iftheangleissmallerthan160°anditisexactlybetween 2 classes, then the angle contributes equally to both the bounds,andthemagnitudeisdividedby2[20].

WhilebuildinganHoG,3subcasesariseasfollows:

inputimages,onefromthedatasetcalledtheactualimage andtheotherfromoutsidethedatasetcalledthecandidate image, and the output is the similarity between the two images.Ifthetwoimagespassedthroughthenetworkhavea smalldistancebetweenthem,theycanbeclassifiedasthe dataset’sactualimage[21].

Fig.1.SiameseNetworkforFaceRecognition

3.2 dlib’s HoG Face Detection

Fig.2.Featurevectorextractioninsisternetworks

Fig.[20].3.

1.)If the angle is smaller than 160° and it is not halfway between2classes,theanglewillbecategorizedintheright categoryofHoG[20].

Finally, a 16x16 block is used to normalize the image, makingitinsensitivetothingslikelighting.Thevalueofthe 8x8sizedHoGisdividedbytheL2 normoftheHoGofthe 16x16blockthatcontainsit,whichisavectoroflength36. Thefeaturevectoriscreatedbyconcatenatingallofthe36x1 vectorsintoasinglelargevectorthatcanbeusedtotrainan SVM(SupportVectorMachine)classifierandusedforface detectionusingthedliblibrary[20].

Kernelsappliedtocomputegradients[20]

Volume: 09 Issue: 03 | Mar 2022 www.irjet.net

These images are passed through similar networks called sister networks, which are similar in terms of their parametersandsharedweights,inordertotraintheneural networktolearnhowtocomputesimilaritiesbetweentwo images,theactualimageandcandidateimage.Thesesister networksaremadeupofaseriesofconvolutional,pooling, andfullyconnectedlayersthatproduceafixed sizefeature vector denoted by h1 as an encoding of the actual image Image1.Thedifference h2 h1 betweentheencodingsof the two images passed is the distance between their encodings.Thevalueof h2 h1 isrelativelysmallifthe twoimagespassedareofthesameperson.Theworkingsof sisternetworksaredepictedinFigure2.

Histogram ofOrientedGradients(HoG) isanabbreviation forHistogramofOrientedGradients.HoG’smainideaisto turnfacialfeaturesfromanimage(orareal timevideo)into avectorandfeeditintoaclassifierlikeSVM(SupportVector Machine)todetectthepresenceofafaceinanimage.The histogramsofdirectionsofgradients,ororientedgradients

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

3.)Iftheangleisgreaterthan160°,thepixelisconsideredto contributeproportionallyto160°and0°[20].

After gradient computation, the image is divided into 8x8 cells to create a compact representation, making our HoG more noise resistant. Then, for each of these cells, HoG is calculated. The gradient’s direction inside a region is estimated by creating a histogram from the 64 gradient directions and their magnitudes within each region. The histogramisdividedintoninecategoriesthatcorrespondto anglesrangingfrom0to180degrees.Thetemperaturesare 0°,20°,40°,80°,and160°[20].

p ISSN: 2395 0072

oftheimage,arethenamesgiventotheseextractedfeatures. Gradientsarelargearoundedgesandcornersingeneral,and theyallowustodetectregionsofinterest(ROI)[20].This methodfordetectinghumanbodieswasdevelopedbyDalal et al. [19]. The images are first preprocessed by being cropped and scaled to the appropriate size. The image gradients must be calculated as the first step in face detection.Thesegradientsarecalculatedtoremoveallnon essentialelementsfromanimage,suchasbackgroundnoise, leavingonlytheregionofinterest(ROI).Kernelsareusedto computethehorizontalandverticalgradients,asshownin fig.3.

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Fig.6.ConceptualArchitectureoftheSystem

Thisprocesscanbeprimarilydividedinto4stages:

4. CONCEPTUAL ARCHITECTURE

4.1 Image acquisition for dataset creation

Fig.5.Subcase3ofHoGbuilding[20]

imagesarefirstcroppedinpreprocessingsothatonlythe Region of Interest (ROI) is available for further detection. Afterthat,thecroppedimagesareresizedtoaspecificpixel position. After resizing the images, the cv2 module of the OpenCVlibraryisusedtoconvertthemfromBGR toRGB. Finally,theprocessedimagesaresavedinthedatasetalong withthestudent’sname.

Ourmethodentailsassessingstudentattendanceusingonly one image per student from the class, captured using a webcamconnectedtoalaptopordesktopcomputer.Allof the students in the class must register on the device by enteringtheirinformation,andeachstudent’simagewillbe capturedandsavedinthedataset.Thestudentwillbeasked toregistertheirattendanceusingthedevicethatthesystem is running on before each lecture. The system will then detect the face, calculate the encodings of the face, and comparethemtothoseinthedataset.Thestudentwill be marked present for the lecture if there is a match. This attendanceinformationwillbesavedinaCSVfilethatthe class’sfaculty/lecturercaneasilyaccess.

We used images of 50 students from our own college to create our dataset. These photographs only show the studentsfrontalfaces.Onlyoneimageisusedperstudent. Afterthat,theimagesinthedatasetarepreprocessed.The

Volume: 09 Issue: 03 | Mar 2022 www.irjet.net

p ISSN: 2395 0072

Fig.4.Subcase2ofHoGbuilding[20]

4.4 Attendance Generation

Volume: 09 Issue: 03 | Mar 2022 www.irjet.net

Fig.9.128 valuedfaceencodingvectorarray

TherecognizedfacesarethenmarkedpresentonaCSVfile, whichcanbegeneratedandassessedinasoftcopyformat onExcel,followingtherecognitionprocess.

Real time image processing and detection are involved in thisstep.Thestudent’sface isdetectedandthestudent is recognized using a webcam to record live video of the student.Becausetheimageiscapturedinrealtime,image distortion can occur if a student is not fully facing the camera.Facelandmarkestimation[15]isusedtodetectthe pose of the face, which solves the problem. There are 68 distinct landmarks on every face. The top of the chin, the outside edge of each eye, the inner edge of each eyebrow, andotherlandmarkscanbefound.

By rotating, scaling, and shearing the face image, these landmarkscanbeusedtocenterit.Thisimagecannowbe usedtocalculatefaceencodings,whicharethencomparedto encodingsalreadystoredinthedatabase,andthestudentis identifiedassuch.

p ISSN: 2395 0072

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Fig.7.ModularDiagramofthesystem

4.2 Face Detection

Theface recognitionmodel[12],whichisheavilybasedon dlib,isusedforfacedetection.Thecolorandsizeoftheslant eyes,thegapbetweentheeyebrows,thedistancebetween the lips and the chin, and other details are noted in this model.Whenallofthesevaluesareaddedtogether,aface encodingiscreated,whichisavectorarraywith128values. Thismodelisloopedthroughthedatasetinoursystemto calculatethefaceencodingsofeachimage.Inthenextstepof face recognition, this face encoding aids in identifying the students.128 valuedfaceencodingvectorarray.

Fig.10.68landmarkspresentontheface

Fig.11.AttendanceListasobservedusingGoogleSheets

4.3 Face Recognition

5. IMPLEMENTATION SETUP

e ISSN: 2395 0056

7. CONCLUSIONS

Accordingtotheimplementationsetup,thematchindexis the smallest difference between the face encodings of the student’s face in the live video capture and those in the database.Figure13depictsthesystem’simplementationas it is being worked on using real time video capture. The matchindex’sconfidencethresholdissetto0.6bydefault.If thematchindexvalueislessthanthisconfidencethreshold, thefacewillberecognizedandidentifiedasthesame.Figure 14showstherelationshipbetweentheliveimagecapture’s confidencescoreandthefacedistance(i.e.matchindex)of theimagesinthedataset.Afterthat,thestudent’sattendance isrecordedinaCSVfile.Ontheattendancesheet,onlythe namesofstudentswhosefaceswerescannedandrecognized will be written. Any app that supports CSV files, such as Excel, Google Sheets, or Numbers, can then be used to evaluatethisfile.

StudentsandfacultycanuseourPyQT basedGUItointeract withthesystem.Whenthestudentsopentheapp,theywill betakentoascreenwheretheycanseethelivevideofeedas wellasthedateandtimewhentheattendanceistaken.To begintheirattendanceprocess,thestudentmustfirstclock in. If the system recognizes the student’s face when they clockin,alabelwiththestudent’snameandthematchindex willbegeneratedaroundtheface.

p ISSN: 2395 0072

Fig.12.GUIofapplicationcapturingreal timevideo

[1] T.FadelelmoulaandAlmaarefaColleges,Riyadh,Saudi Arabia, “The impact of class attendance on student performance,”Int.Res.J.med.Med.Sci.,pp.47 49,2018.

6. RESULT AND ANALYSIS

Volume: 09 Issue: 03 | Mar 2022 www.irjet.net

Fig.13.LivewebcamcaptureoftheStudentswith Identification.Thematchindexismappedtotheidentified student’slabelinoursystem.

Thesystemthatwehavedevelopediscurrentlyviablefor individual classroom attendance. With the required enhancementsandcreationofaproperdatabaseconsisting ofallthedetailsofeachstudentinthecollegeoruniversity, itcanbewidelyusedatthecollegiatelevel.Thissystemcan alsobeusedtomanagethestudentsofnotonlythestudents butalsothefacultymembers,staff,andnon staffmembers aswell.Anotherdevelopmentwhichwewishtoensureisthe complete automation of the system. Currently, the system has to be supervised to avoid any discrepancies such as tampering with the devices. Our goal is to completely automatetheprocessbyusingareal timelivefeedcapture using a CCTV camera, which can mark the attendance of studentswithoutanymanualsupervision,therebyproducing legitimateanduntamperedattendancereports.

REFERENCES

Individualclassroomattendanceiscurrentlyfeasibleusing the system we developed. It can be widely used at the collegiate level with the necessary enhancements and the creationofaproperdatabasecontainingallofthedetailsof eachstudentinthecollegeoruniversity.Thissystemcanbe usedtomanagenotonlystudents,butalsofaculty,staff,and nonstaffmembers’students.Anotherdevelopmentthatwe wanttomakesureofisthesystem’scompleteautomation. To avoid any discrepancies, such as tampering with the devices,thesystemmustcurrentlybesupervised.Ourgoalis tocompletelyautomatetheprocessbyusingareal timelive feed captured by a CCTV camera to mark students’ attendance without the need for manual supervision, resultinginlegitimateanduntamperedattendancereports.

International Research Journal of Engineering and Technology (IRJET)

8. FUTURE SCOPE

[4] J. W. S. D’Souza, S. Jothi, and A. Chandrasekar, “Automated attendance marking and management systembyfacialrecognitionusinghistogram,”in2019 5thInternationalConferenceonAdvancedComputing CommunicationSystems(ICACCS),2019,pp.66 69.

[13] https://medium.com/@ageitgey/machine learning is fun part 4 modern face recognition with deep learning c3cffc121d78

[14] https://machinelearningmastery.com/one shot learning with siamese networks contrastive and triplet loss for face recognition/

p ISSN: 2395 0072

[5] J. Harikrishnan, A. Sudarsan, A. Sadashiv, and R. A. S. Ajai, “Visionface recognition attendance monitoring systemforsurveillanceusingdeeplearningtechnology andcomputervision,”in2019InternationalConference onVisionTowardsEmergingTrendsinCommunication andNetworking(ViTECoN),2019,pp.1 5.

[16] https://www.softwaresuggest.com/blog/manual vs automated attendance system pros cons/

[6] T.A.Kiran,N.D.K.Reddy,A.I.Ninan,P.Krishnan,D.J. Aravindhar, and A. Geetha, “PCA based Facial Recognition for Attendance System,” in 2020 International Conference on Smart Electronics and Communication(ICOSEC),2020,pp.248 252.

[11] recognition/enhttps://www.electronicid.eu/en/blog/post/face

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

[3] S.Sawhney,K. Kacker,S. Jain,S.N.Singh,andR.Garg, “Realtime smart attendance system using face recognition techniques,” in 2019 9th International Conference on Cloud Computing, Data Science Engineering(Confluence),2019,pp.522 525.

[17] https://www.edusys.co/blog/rfid attendance system

[19] N.DalalandB.Triggs,“Histogramsoforientedgradients forhuman detection,”in2005IEEEComputerSociety ConferenceonComputerVisionandPatternRecognition (CVPR’05),2005,vol.1,pp.886 893vol.1.

[20] detection/1https://maelfabien.github.io/tutorials/facetheory1

[7] M.S.Akbar,P.Sarker,A.T.Mansoor,A.M.AlAshray,and system,”J.Uddin,“FacerecognitionandRFIDverifiedattendancein2018InternationalConferenceon Computing, Electronics Communications Engineering (iCCECE),2018,pp.168 172.

[9] S.SaypadithandS.Aramvith,“Real timemultipleface recognition using deep learning on embedded GPU system,” in 2018 Asia Pacific Signal and Information ProcessingAssociationAnnualSummitandConference (APSIPAASC),2018,pp.1318 1324.

[15] V. Kazemi and J. Sullivan, “One millisecond face alignment with an ensemble of regression trees,” in 2014IEEEConferenceonComputerVisionandPattern Recognition,2014,pp.1867 1874.

[10] F. Deeba, H. Memon, F. Ali, A. Ahmed, and A. Ghaffar, “LBPH basedenhancedreal timefacerecognition,”Int.J. Adv.Comput.Sci.Appl.,vol.10,no.5,2019.

[8] Smitha, P. S. Hegde, Afshin, and Yenepoya Institute of Technology, “Face Recognition based Attendance Management System,” Int. J. Eng. Res. Technol. (Ahmedabad),vol.V9,no.05,2020.

Volume: 09 Issue: 03 | Mar 2022 www.irjet.net

[18] S. K. Baharin, Z. Zulkifli, and S. B. Ahmad, “Student absenteeismmonitoringsystemusingBluetoothsmart location based technique,” in 2020 International ConferenceonComputationalIntelligence(ICCI),2020, pp.109 114.

[21] https://heartbeat.comet.ml/one shot learningpart12 definitions and fundamental techniques 1df944e5836a

[2] A. G. Menezes, J. M. D. da C. Sa, E. Llapa, and C. A. EstombeloMontesco, “Automatic attendance managementsystembasedondeeponeshotlearning,” in 2020 International Conference on Systems, Signals andImageProcessing(IWSSIP),2020,pp.137 142.

[12] https://pypi.org/project/face recognition/

Turn static files into dynamic content formats.

Create a flipbook