InternationalJournalofComputer ScienceandEngineering(IJCSE)
ISSN(P):2278–9960;ISSN(E):2278–9979
Vol.11,Issue2,Jul–Dec2022;1–8 ©IASET
BASEDAUTOMATICPERSONALITYRECOGNITIONUSEDINASYNCHRONOUS VIDEOINTERVIEWSOFSTRESSDETECTIONUSINGFACEIMAGESANDFACIAL LANDMARKBYUSINGTHECONVOLUTIONNEURALNETWORK(CNN)ALGORITHM
PriyankaH1&BLJayakumar2
1ResearchScholar,DepartmentofComputerScience&Engineering,SEACollegeofEngineering&Technology, KRPuram,Bengaluru,India
2AssistantProfessor,DepartmentofComputerScience&Engineering,S.E.ACollegeofEngineering&Technology, K.RPuram,Bengaluru,India
ABSTRACT
Withthehelpoffacephotosandfaciallandmarks,wesuggestastressrecognitionalgorithminthiswork.Adevicefor gatheringthenecessarydataisneededalongtheeventofstressdetectionutilisinganaturalorbiologicalsignalor thermalpicture,thusbeingimportantareaforresearch.Toaddressthisflaw,weputforthanalgorithmthatcanidentifya person'sbehaviourfromstillvideosorphotostakenusinganormalcamera,includingcreatingin-depthneuralnetwork, usesfacialidentificationsinfusedalongbenefitingthatwhensomeoneisbeingstressed,theireye,mouth,andhead movementsdifferalonghowtheynormallybehave.Likewise,byidentifyingacandidate'sbehaviourduringanonline interview,wecandeterminewhetherornottheyarequalified.Thesuggestedalgorithmrecognisesbehaviourmore accurately,accordingtoexperimentaldata.
KEYWORDS:C++,Python,Java,ConvolutionalNeuralNetwork(CNN),PersonalityRecognition,OpenCV,HAAR CascadeandMatLab,OpenCV,Espeak,Xming&Putty
ArticleHistory
Received:06Jul2022|Revised:07Jul2022|Accepted:07Jul2022
1.INTRODUCTION
Inmanysituationswheremoresecurityorpersonaldataaboutthepersonisneeded,humanemotiondetectionisused.We couldneedtosetupasecondlayerofprotectionwhere,inadditiontotheface,theemotionisalsodetected.Thiscanbe consideredasthesecondstageafterfacedetection.Asystemiscurrentlybeingdevelopedtodetectwhenauserisunder stressandtoprovidefeedbackintheaimofreducingstresswhenunderstress,asmodernindividualsexperience tremendouslevelsofstress.Additionally,wesuggestedthatthestudent'semotionsbeacknowledgedandthattheworried teacherreceiveanupdate.Focusingonknowingclassesforstudentsincludingtime-tablebeingsharedamongstthem.
Intheseinvestigations,characteristicsfrombiosignalssuchtheelectrocardiogram,electrodermalactivity, respiration,galvanicskinresponse,andheartratevariabilitywereextractedandusedtoexpressstress.Furthermore,alot ofthemmadeuseoftraditionalclassifierslikeSupportVectorMachine,LinearDiscriminantAnalysis,Adaboost,andKNearestNeighbor[1].
www.iaset.us
editor@iaset.us
Alongthevicinityofrecognisingstresses,numerousmethodsthrubio-signalsbeingevaluated,thoughameasure andalongbio-signalling,anend-usermayexperiencerejectionsduetotoolsappliedinmeasuringbio-signallingthusbeing rootedalongthebody.Thusnumerousobservationsalongstress-recognitionthruthermalimagingweredone,thusevena demeritalongcomplicationsinrecognizingstresssimplyalongeverydaylifeduetounrecognisablewithoutanytoolsof thermalimaging,parallellyintheventofstress-recognitionobservationsthrucommonimaging,popularevaluations adaptedbeingconsiderablyfeaturedassimple.
Inthisarticle,weprovideatechniqueforidentifyingstressthatinvolvesextractinghigh-dimensionalfeatures fromfacephotostakenwitharegularcamera.Additionally,weusetheplacementoffacelandmarksthatsignificantlyvary whenunderstressinordertolearnmoreeffectivefeatures.
2.RELATEDWORK
Physiologicalparameterevaluation,bodilyfluidmeasurement,andthepapermethod(self-report)aresomeofthe behaviouranalysistechniquesnowinuse.Inthepaperapproach,individualsaregivenamultiple-choicequestionnaireto complete,andeachresponseisassignedaspecificscore.Aftertherespondentcompletesthequestionnaire,theresultsof eachchoicewillbeaddeduptoproduceafinalscorethatrepresentstheindividual'slevelofconduct.Fortheidentification ofthestresshormone,Cortical,bodyfluidtestssuchasbloodorsalivaareperformed.Thesetwoapproachesare ineffectivefortrackingstressovertime.Monitoringandanalysingphysiologicaldatacangiveoneausefulunderstanding oftheirhealth.
BehaviorRecognitionusingBio-Signals
Becausetheydisplaythebody'smostsensitivealterationsandenabletheidentificationofchangesinthebodythatarenot indicatedbyfaceandbehaviour,bio-signalswereusedinearlyresearchonstressrecognition.Intheseinvestigations, characteristicsfrombio-signalssuchtheelectrocardiogram,electrodermalactivity,respiration,galvanicskinresponse,and heartratevariabilitywereextractedandusedtoexpressstress.AndalotofthemmadeuseoftraditionalclassifierslikeKNearestNeighbor,AdaBoost,SupportVectorMachine,andlineardiscriminantanalysis.
BehaviorRecognitionUsing
Thermalpicturemanystudieshavebeendonetodetectthechangebyusingthethermalimagingtodetectthechange becausewhenapersonisstressed,thebloodflowandtemperatureofthefaceincrease.Thisresearchusedavarietyof techniquestoidentifystress,includingdirectlyextractingcharacteristicsfromthermalpicturesandextractingfeatureslike respirationrate,blinkfrequency,skintemperature,andbloodflowfromthermalimages.
BehaviorRecognitionusingGeneralImage
Eye,lip,andheadmovementschangewhenapersonisstressedcomparedtonormalcircumstances,andresearchonstress recognitionusingcommonimageryisalsobeingdone.Theseresearchemployedavarietyoftechniquestoidentifystress, includingtheextractionofhand-craftedfeaturesfromthenose,mouth,andeyeregionsaswellastheuseofeyesize,lip motions,andheadmovementsasfeatures.
BasedAutomaticPersonalityRecognitionUsedinAsynchronousVideoInterviewsofStressDetectionAlgorithm 3
UsingFaceImagesandFacialLandmarkbyUsingtheConvolutionNeuralNetwork(CNN)
3.LITERATUREREVIEW
HumanAttitude-RecognitionApplicationsBasedonFacialFeaturethroughFaceDetection
Author:ArdinintyaSetyadiDivaet.al,
Publishedin:2015InternationalElectronicsSymposium(IES)
Abstract
Human-psychology,exploredalong4basicpersonalitieslikesanguine,choleric,melancholic,andphlegmatic.Basicmode ofknowinghumanfundamentalpersonalityisbytesting,thusbeingGraphotest(handwritingtest).Currentstudyitsbeing executedondetectinghumanfundamentalpersonalitythrucollectivefeaturedfaces:theeyes,lips,andnose(priortotest), thusgotthrureceivedimagesoffaces,gapamongstthecornersoftheeyes,ratiosamongstmouthwidthandnose,ratiosof widthamongsteyes,lipthicknessbeingextractingfeatures,thruANN:artificialneuralnetworks(back-propagation),also relayingonextractingsuchfeatures,thefundamentalpersonalitybeingidentifiedConsideringpracticalfindings,system detectshumanfundamentalpersonalityalongsimilarinputimagingdataalongtrainingaverageratios85.5%.The identificationfindingsvariousinputimagingdataalongtrainingbeingweighedas42.5%average,situationsdemandsthru identificationofpersonalityalongcholeric,phlegmaticreadslessenedabout50%ratios.
ADVANTAGE
Theyhaveimplemented2typesoftesting,oneonthesamephotoandanotheronthedifferentphoto.
DISADVANTAGE
Testingaccuracybeingminimallyabout50%alongfewfeaturesonalimitedoccasions
SentimentClassificationandPersonalityDetectionviaGalvanicSkinResponseBasedonDeepLearningModels
Author:TaoHong;XiaoSun;FangTian;FujiRen
PublishedIn:20195thInternationalConferenceonBigDataComputingandCommunications(BIGCOM)
Abstract
Sentimentandpersonalityhaveasignificantinfluenceonhowwethink,create,andmakedecisionsinourdailylives. Manymethodshavebeenputforthtoautomaticallydetectusers'sentimentinspeechandimage.Forsomecircumstances, suchasinterviewsandpolygraphs,beingabletoaccuratelyforecastaperson'semotionsandpersonalitytraitscanbe helpful.Inthiswork,severalmodels,suchasJointLearningModelofConvolutionalNeuralNetworkandmemoryalong long-Short-Term,alsospatiotemporalhybridmodelswereprojectedtoknowautomaticallyaboutgalvanicskinresponse (GSR),alsovideoclipsratingagainstsimilartosentimentsrecognition,personalitydetectionfacts,matchingalong projectedmodels,alsostate-of-the-artmodelsalongsimilarworks.Thepracticalfindingspresentedagainstpredominant findingsalongsentimentrecognition,alsopersonalitydetectionbeingbenefittedalongprecision,recall,andF1score.
ADVANTAGE
MatchedagainstothersimilartrialsadaptingGalvanicSkinresponse(GSR)signalalongemotionalcategorisation ofpersonalitydetection,practicalfindingsareenhanced.
www.iaset.us editor@iaset.us
DISADVANTAGE
Thecategorisationalongsurprise,happinessbeingatoughfact
StudyonDeterminingtheBig-FivePersonalityTraitsofanIndividualBasedOnFacialExpressions
Author:MihaiGavrilescuPublishedIn:2015E-HealthandBioengineeringConference(EHB)
Abstract
Accordingtopreviousstudies,thereisanincreasingdemandforinformationaboutpeople'spersonalitiesandbehavioursin areaslikecareerdevelopmentandcounselling,individualisedhealthcare,mentaldisorderdiagnosis,andtheearly detectionofphysicalillnesseswithpersonalityshiftsymptoms.TheBig-Fivepersonalitytypescancurrentlyonlybe identifiedbycompletingaquestionnaire,whichtakesanunreasonableamountoftimeandcannotbeusedfrequently.Our researchintendstodevelopacutting-edge,non-invasivemethodforidentifyingtheBig-Fivepersonalitytraitsbasedon facialfeaturescollectedwiththehelpoftheFacialActionCodingSystem.Thefindingsindicatearelationshipbetweena person'spersonalityqualitiesandtheFACSactionunitspresentinfacefeaturesattheirhighestintensitiesAdditionally, comparedtotherequiredforcompletingaquestionnaire,thesystemdevelopeddeliversover75%accuracyofthe predictionopennesstoexperience,aswellastraitanxietyandextraversion,anditispractical,givingfindingsinlessthan3 minutes.
ADVANTAGES
Approximatetimetakenbythesystemtoprovideresultswasroughly45secforvideosequenceswithincluded emotions.
DISADVANTAGE
Unabletoaccuratelydetermineconscientiousnessandagreeablenesswhicharepartofpersonalitytraits.
4.IMPLEMENTATION
I.Proposedalgorithm
inthispartweprovideanapproachtoenhancetheperformanceofstressrecognitionoverallstructureinthesuggested approachstressrecognitionbeginswithfacepictureandfaciallandmarkdetectioninordertodetectfacesmorecorrectly weemployadeeplearningtechniquethatusesthreenetworkssequentiallyweemployacustomapproachthatemploysa cascadetypeofextractedfeaturesalongsudden-femalso,theregressiontreeclassifieralongrecognisingface identificationsgraphicbelowvisualisescollectiveframeworksflowchart
ImpactFactor(JCC):8.5226
NASRating:3.17
BasedAutomaticPersonalityRecognitionUsedinAsynchronousVideoInterviewsofStressDetectionAlgorithm 5
UsingFaceImagesandFacialLandmarkbyUsingtheConvolutionNeuralNetwork(CNN)
Alongtheestimatednetwork,thefacialimaging,identifiedpreviouslyisinputtooutputstressrecognition findings.Thestructureoftheestimatednetworkisvisualisedalongfig.
Progressednetwork,adaptedshortcutassigning,bottleneckarchitecturetoenhanceneuralnetworkstructure. Throughthisshortcutassigningtotheneuralnetworkstructuresupportedalongnumerouslayers,encouragesin simplifyinglearningmethods,alosexplainslearningdirectionsMakingitsuccessfulinsimplyenhancingthein-depth neuralnetwork,alsotoprogressaccuracyalongprogresseddepth.Thruimplyingbottleneckarchitecture,thequantumof internalfactorsbeinglessenedalongelevatingquantumoffeaturemaps,whichelevatesfunctionality,declinesquantumof computation.
II.Input/OutputDesign
Systemdesignshowstheoveralldesignofsystem.Inthissectionwediscussindetailthedesignaspectsofthesystem:
ImageCapture
Staticphotosorimagesequencesareutilisedtodetectfaceexpressions.Acameracanrecordpicturesoffaces.Face recognition
Inordertoidentifyfacialimages,facedetectionishelpful.Facedetectionisdoneinthetrainingdatasetusingthe OpencvimplementationoftheHaarclassifierVoila-Jonesfacedetector.Thevalueofafeaturewithahaar-likestructureis thedifferenceinthesumofthepixelvaluesintheblackandwhiteregions,anditencodesthevariationinaverageintensity indifferentregionsoftheimage[6].Imagepreparation
Noiseiseliminatedduringimagepre-processing,andbrightnessandpixelpositionvariationsarenormalised.
ColorNormalization\s
HistogramNormalization
ExtractionofFeatures
Thechoiceofthefeaturevectorinapatternclassificationissueiscrucial.Afterpre-processing,thefacialimageisusedto extractthekeyfeatures.Scale,attitude,translation,andfluctuationsinilluminationlevelaresomeofthefundamental issueswithimageclassification[6].ClassificationClassificationisusedtoreducethehighdimensionalityofdatathatwas obtainedusingthefeatureextractionmethod.Convolutionalneuralnetworkalgorithmwillbeusedtoclassifyobjects,and featuresshouldtakedistinctvaluesforobjectsbelongingtodifferentclasses.
www.iaset.us editor@iaset.us
Figure2. Database Imagesof different Face detection from Image Extarct EmotionIII.DataFlowDiagram
SequenceDiagram
SequenceDiagram
Inasequencediagram,objectconnectionsarearrangedintemporalorder.Itillustratestheclassesandobjectsinvolvedin thesituationaswellasthemessageflowthatmustoccurfortheobjectstofunctionasintended.Sequencediagramsand usecaserealisationsarefrequentlyconnectedinthelogicalviewofthesystemunderdevelopment.Othernamesfor sequencediagramsareeventdiagramsandeventscenarios.
ActivityDiagram
Thesystem'sdynamicfeaturesaredescribedintheactivitydiagram,aUMLdiagram.Inactuality,aflowchartcontrols howeacheventproceeds.Thesystem'soperationcanbecharacterisedastheevent.Thecontrolflowmustbefollowed betweentasks.
ImpactFactor(JCC):8.5226
NASRating:3.17
UsingFaceImagesandFacialLandmarkbyUsingtheConvolutionNeuralNetwork(CNN)
5.REQUIRMENTS
A.Functionalrequirements
Asystem'sfunctionalrequirementsspecifywhatthesystemshouldbeabletoachieve.Thesespecificationsaredetermined bythekindofsoftwarebeingcreatedandtheintendedaudience.
CreatingWebbasedfunctionsforTraitanalysis.
Designing/formulatingquestionsforanalysis,visualisingtousersequentially
Capturingfacialimagingthruansweringtoquestions.
Analyzingtraitsalongimaging
Visualisingclarifiedassessmentreport.
B.Non-FunctionalEssentialities
Nonfunctionalessentialitiesarethosethatdonotdirectlyrelatetothesystem'sperformanceofthegivenfunction.They mightbeconnectedtoemergentsystemattributesincludingdependability,responsetime,andstoreoccupancy.
C.HARDWAREANDSOFTWARENECESSITIESHARDWARE
Figure6.
6.RESULTS
Considerableobjectivesalongthisworkbeingdesigninganefficient,accuratealgorithm,thusidentifiesattitudeanalysis alongtheinterviewaspirants,attitudeidentificationofaspirantsSupportsaspirantswhoareunabletoparticipatein interviewoncompanyvicinityBenefitsalongsavingtime,manpowerofinterviewer.Alongfacialidentificationto functioncapably,itsnecessarytoavailanimaginginputthatshouldnotblur/printedHereadaptedalgorithmalongfacial identification,featureextractions,systemgeneratesautomaticquestionnairesagainstaspirantsavailablealongcomputer, alsoidentifythepersonalityofaspirantalongthemodeofansweringthequestions.Feedbackbeinggenerated automatically,thusreceivedofinterviewersmailbox.Functionalrealtimeanalysis,probabilitydatarepresentations
7.CONCLUSION
Wesuggestastressrecognitionsystemthatmakesuseoffacelandmarksandfacephotos.Theexperiment'sfindings showedthatemployingfacelandmarksenhancedtheperformanceofstressrecognition.Becausetheymakeiteasierfor youtocomprehendeye,mouth,andheadmovements,faciallandmarksarebetterathelpingyoudetectstress.Wealso discoveredthatwhenemployingagreyfacialimageoftherightsize,performancewasenhancedbymoreaccurately identifyingstress-relatedinformation.
www.iaset.us editor@iaset.us
Futurestudiesaimtoenhancetheperformanceofstressrecognitionbyutilisingtime-axisinformationonhead, mouth,andeyemotion.
REFERENCES
1.Sriramprakash,S.,Prasanna,V.D.,&Murthy,O.R.(2017).Stressdetectioninworkingpeople.Procedia computerscience,115,359-366.
2.Mohd,M.N.H.,Kashima,M.,Sato,K.,&Watanabe,M.(2015).Mentalstressrecognitionbasedonnon-invasive andnon-contactmeasurementfromstereothermalandvisiblesensors.InternationalJournalofAffective Engineering,14(1),9-17.
3.Prasetio,B.H.,Tamura,H.,&Tanno,K.(2018,May).SupportVectorSlantBinaryTreeArchitectureforFacial StressRecognitionBasedonGaborandHOGFeatureIn2018InternationalWorkshoponBigDataand InformationSecurity(IWBIS)(pp.63-68).IEEE.
4.Zhang,K.,Zhang,Z.,Li,Z.,&Qiao,Y.(2016).Jointfacedetectionandalignmentusingmultitaskcascaded convolutionalnetworks.IEEESignalProcessingLetters,23(10),1499-1503.
5.Kazemi,V.,&Sullivan,J.(2014).Onemillisecondfacealignmentwithanensembleofregressiontrees.In ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition(pp.1867-1874).
6.He,K.,Zhang,X.,Ren,S.,&Sun,J.(2016).Deepresiduallearningforimagerecognition.InProceedingsofthe IEEEconferenceoncomputervisionandpatternrecognition(pp.770-778).
7.Nishanth,N.,AndKamalKhumarLs."AnAutomaticPersonalityRecognitionSystemUsingTimeDistributed Cnn."InternationalJournalofComputerScienceEngineeringandInformationTechnologyResearch(IJCSEITR)
ISSN(P):2249
6831;ISSN(E):2249
7943Vol.12,Issue1,Jun2022,1-12