HAND GESTURE VOCALIZER

e ISSN: 2395 0056

ISSN:

HAND GESTURE VOCALIZER

DICHOLKAR5

0072

1,2,3,4 Students, Department of Electronics and Telecommunication, ATHARVA COLLEGE OF ENGINEERING, Malad Marve Road, Charkop Naka, Malad (West), Mumbai 95

5Professor, Department of Electronics and Telecommunication, ATHARVA COLLEGE OF ENGINEERING, Malad Marve Road, Charkop Naka, Malad (West), Mumbai 95

***

Abstract Hand Gesture Vocalizer is a social purpose projectthathelpspeopletoupliftwhoarespeech,blindand hearing impaired, by facilitating them to have a better communicationwiththepublic.Invariousresearchesithas beenestimatedthatthereareabout9.1billionpeopleinthe worldwhoaredeaf,blindandarehavespeechimpairments andtheyfacealotofproblemswhilsttryingtocommunicate withsocietyintheirdaytodaylife.Generally,signlanguage is used by the deaf, blind and ones who have speech impairments, but this becomes difficult for them to communicatewithotherpeoplewhodonotunderstandsign language.Signlanguagewhichisusedtoconveyinformation between people relies various body language, orientation andmovementsof thearm andfingers etc.This project is designed to address the need of developing an electronic device that can translate sign language into speech and displayinordertoremovethecommunicationgapbetween thedumbandblindandthegeneralpublic.

Keywords Gesture detection, flex sensors, tilt detection

Introduction

Generally sign language is used by the blind, speech and hearing impaired people for communication but they find difficulty in communicating with others who don’t understand sign language as Sign Language involves orientationandmovementofhands,arms/body,combining of various hand shapes and various facial expressions to fluidly express the thoughts. As the sign Language is generallynotauniversallanguage.Forsignlanguagethere doesnotexistasinglestandard,universallyacceptedscheme .Sothemainaimsistolowerthecommunicationbarrierand make adevicethatcantranslatesignlanguageintoaudio and visual form in order to make the communication betweenthemutecommunitiesandthegeneralpubliceasily . So, the main objective is to make a simple prototype by taking some of the gesture and convert it into audio and visualformsothattheycanunderstandbyeveryone.

It is designed to facilitate the communication among the dumb,deafandblindcommunitiesandtheircommunication withthenormalpeople.

AmicrocontrollerbasedHandGestureVocalizersystem which makes use of flex sensors and an accelerometer for gesturedetectionandthetiltdetectionrespectively.Inthis theinputvaluesthataregivenorcorrespondtoaparticular gesturearethenplayedasavoicemessagethroughaandroid phone using Bluetooth module as well as text on the LCD screendisplay

Literature review

Kunal Kadam, Rucha Ganu, Prof. S. D. Joshi developed AmericanSignLanguageInterpreter Asimpleprototypeto check the recognizing sign language using smart sensor gloves. The system was not accurate as the time taken to ensure recognition of sign was more [1]. Srinivas Gutta, JeffreyHuang,showedtheadvancesinthemethodologyof hybridclassificationarchitecturesforfaceandhandgesture recognitiontasks.Thissystemishighlyefficientbutfailsin circumstances where images are blurred or there isn’t enough light to efficiently identify the gesture [2]. R.L.R LokeshBabu,S.GaganaPriyanka,P.PoojaPravallika ,Ch. Prathyusha made “Gesture Vocalizer”.. It uses Real Time SpeechtoTextConverterforMobileUsers.Butthisbecomes difficult to always look into mobile phone application [4] Prerana K.C, Nishu Mahato propose a method of Hand Gesture Recognition system. A system that can read the valuesforaparticulargesturedonebytheuserandpredict theoutputforthegesture. Thissystemisnotthatreliable becausethedatasetforallthealphabetsandfrequentlyused words was not created [5]. M. S Kasar, Anvita Deshmukh madeaSmartSpeakingGlove.Aglovewithportablespeech synthesizer and vocalizer for the speech impaired people. The system developed was good but capable of detecting only4differenthandgesturesandconvertthemintovoice outputs[7].

Block diagram

Hardware implementation

GESTURE DETECTION:

Theglove is equipped with three flex sensors.Thesensors are placed on the three fingers of the hand glove. These sensorsareanalogsensors.Theycanbemadeunidirectional orbidirectional Eachandeverybendorevenalittlebend of the finger can bedetectedbythesensors.Thebendingof each finger can be obtained into 10 different levels. The finger must be at one of these10levelsatanystageand it can easily determine how muchthe finger isbent.After thisprocessfromtheflexsensorthebinarydataisthensent to ATMEGA328. The next step is to combine the each movementofthefingerandthennameitaparticulargesture ofthehand

Theinputistakenfromadigitalglovethathas3flexsensor attachedtothegloveateachfinger.Forthebenddetectionof fingers andforgetting the tiltandtheorientation ofhand oneADXL3353 axisaccelerometerisused. Forconverting the hand movements or gestures into audio through an audioprocessingunitandvisualdatathroughanLCDdisplay a microcontroller isused. Thedata of multiple gestures is storedinthecontroller.Thegestureoutputisthensentto theLCDdisplayandthroughBluetoothmoduleitissentto themobile.TheBluetoothmodulethroughphonegivesthe audiooutputofthegesture.

Circuit diagram

TILT DETECTION :

In the Hand gesture vocalizer system an Accelerometer (ADXL 335)used as a tilt detector.ADXL335isknownas a three axis analog accelerometer IC. The X, Y and Z acceleration are readed as analog voltages by the accelerometer.An accelerometer can figureout theangle it is tiltedby measuring the amountofaccelerationdue to gravity. Theaccelerometer can find out howfastand in what direction the device is moving, by sensing the amount of dynamic acceleration.The mainandthebasic function of the accelerometer is detecting thetilting of the hand. After this process it sends some binary data againstmeaningful gestures,to the microcontroller.After thattheMicrocontroller receivesthe data and saves them.

Fig. 1 Block Diagram [5]

Fig. 2 - Circuit diagram

Fig. 3 Interfacing flex sensor with arduino nano

BLUETOOTH MODULE :

HC O5 Bluetooth module is also known as Master/Slave module and by default it is Slave. The role of Blue tooth module configured only by AT Commands. Master can initiateconnectionwithotherBluetoothmodulewhileslave canreceivedata.Itcannotinitiatetheconnectionwithother module.

Software implementation

The software used for the preparation of the project is ArduinoIDE.IntheopensourcesoftwareArduino(IDE)itis easytowritethecodeanduploadittotheboard.

ThissoftwarerunsonWindows,MacandLinux.Thiswasthe softwaretodraftthefinalcodeforthisproject.Thecodeis writteninC++ andisbased onprocessing andotheropen sourcesoftware.

Result

LCD DISPLAY :-

TheoutputfromtheFlex sensor and accelerometer are alsosent to the LCDdisplay.The microcontroller checks each signal which is received and compares it with the already stored value.

Themicrocontrollertakesthedecisionaboutwhatmessage shouldbedisplayed,onthebasisofthiscomparison After the decision is made, the microcontroller sent an 8 bit addresstotheLCD.Thereceived8 bitaddressisthelocation offirstcharacterinthemessagethattheLCDshoulddisplay

Fig. 4 Interfacing accelerometer with arduino nano

Fig 5 Interfacing accelerometer with lcd

Conclusion

Hand Gesture vocalizer is a gesture based interface for reducing the communication gap and facilitating communication among normal people and people with speech,blindandhearingdisabilities.Anormalgloveisused which is fitted with three flex sensors along the length of eachfinger.Amicrocontrollerandsensorbasedgestureto voiceanddisplay converteriscreated inthisproject.This canbehelpfultorecognizethecommonlyusedgesturesand convertthemintotheformvoicemessageaswellasatext message for the benefit of the blind, speech and hearing impairedpersons

A microcontroller based Hand Gesture Vocalizer system whichmakesuseofflexsensorsandanaccelerometerfor gesturedetection andthetiltdetectionrespectively,sowith the help of this the input values which correspond to a particular gesture are then played as a voice message throughandroidphoneusingBluetoothmoduleaswellas displayedastextontheLCDscreen.

Future enhancements

HandGestureVocalizerisasocialpurposeprojectthathelps peopletouplift,whoarespeech,blindandhearingimpaired, byfacilitatingthemtohaveabettercommunicationwiththe public. This system makes the communication among the dumb,deafandblindcommunitiesandtheircommunication withthenormalpeopleveryeasy.

Designing a system like wireless transceiver for " Sensors andMicrocontrollerBasedGestureVocalizer".Perfectionin monitoring and sensing of the dynamic movements is involved in Microcontroller and Sensors Based Gesture Vocalizer".Designing a jacket, which would becapable of vocalizingthegesturesandmovementsofanimals.Virtual reality application such as in videogames replacing the conventionalinputdeviceslikejoystickswiththedataglove. The Robot control system to regulate machine activity at remotesensitiveplaces,tele operatorstoperformsurgeries withthehelpofexpertsurgeonsremotelyareanotheruseful applicationsofsuchanimplementation.

International Research Journal of Engineering and Technology (IRJET)

Volume: 09 Issue: 04 | Apr 2022 www.irjet.net

References

1. KunalKadam,RuchaGanu,AnkitaBhosekar,Prof.S.D. Joshi , “American Sign Language Interpreter”, Proceedings of the 2012 IEEE Fourth International ConferenceonTechnologyforEducation.

2. Srinivas Gutta, Jeffrey Huang, Ibrahim F. Imam,and Harry Wechsler,“Face and Hand GestureRecognition Using Hybrid Classifiers”, ISBN: 0 8186 7713 9/96, pp.164 16

3. Hope M Hurlbut, "A Lexicostatistic Servey of theSignedLanguagesinNepal,"SILElectronic Survey Reports,p.23,June2012.

4. R.L.R. Lokesh Babu , S. Gagana Priyanka , P.Pooja Pravallika,Ch.Prathyusha “GestureVocalizer”.

5. PreranaK.C,NishuMahato,andBishrutiSatyal,"Hand GestureRecognitionusingKINECT,"NepalEngineering College,Baktapur,BachelorThesisP3 I.0 14 4,2015.

6. Kiran R,”Digital Vocalizer System for Speech and Hearing Impaired” , International Journal of Advanced ResearchinComputerandCommunicationEngineering Vol.4,Issue5,May2015

7. M. S Kasar, Anvita Deshmukh, Akshada Gavande, and PriyankaGhadage,"SmartSpeakingGloveVirtualtongue forDeafandDumb,"InternaltionalJournalofAdvanced ResearchinElectrical,ElectronicsandInstrumentation Engineering,vol.5,no.3,p.7,Mar.2016.

8. A review on hand gesture system Mr. Prashant Chaudhari,E&TCDepartment,GovernmentCollegeof Engineering, Maharashtra 425001(INDIA) Prof. G. R. Phulay Asst. Prof. in E &TC Government College of Engineering, Jalgaon, Maharashtra 425001(INDIA)Mr. RavindraPatilE&TCDepartmentGovernmentCollegeof Engineering, Jalgaon,Maharashtra 425001(INDIA)(Jan 2013).

9. The amazing digital glove that give voice to voiceless PraveenkumarSHavalagi,ShruthiUrfNiveditaM.Tech (VLSI), Department of ECE, SIET, Bijapur, India, B.E (ECE), Department of ECE, SIET, Bijapur, India(MAR 2013).

10. Innovative Approach for Gesture to Voice Conversion PriyankaR.PotdarJSPM’sBhivrabaiSawantInstituteof Technology&Research(W) Universityof Pune,Pune, Maharashtra, India Dr. D.M. Yadav, Principal, JSPM’s Bhivrabai Sawant Institute of Technology & Research (W)UniversityofPune,Pune,Maharashtra,India(June 2014).

e ISSN: 2395 0056

p ISSN: 2395 0072

11. Jean Christophe Lementec and Peter Bajcsy, “Recognition of Gestures Using Multiple Orientation Sensors:GestureClassification”,2004IEEEIntelligent Transportation Systems Conference Washington, D.C., USA,October34,2004,pp.965 970.

12. Sushmita Mitra and Tinku Acharya, ”Gesture Recognition:ASurvey”.

Factor value: 7.529 | ISO 9001:2008 Certified