Issuu

SMART ASSISTIVE GLASSES FOR SIGN LANGUAGE COMMUNICATION AND NAVIGATION

Mohammed raziuddin qureshi 1 , Tejavath hathiram 2 , Kayitha ravikumar 3 , Dharavath naveen4 , Dr. Himanshu sharma5

1,2,3,4 B Tech final year students, ECE department, JB Institute of Engineering and Technology, Hyderabad, Telangana 5 Associate professor, ECE Department, JB Institute of Engineering and Technology, Hyderabad, Telangana ***

Abstract –

Smart Assistive Glasses for Sign Language

Communication and Navigation is an embedded system developed to improve communication and mobility for individuals with hearing and speech impairments. The system uses a Raspberry Pi 4 as the core processing unit, integratedwithaPiCamera,GPSmodule,ultrasonicsensors, speaker, and USB sound card to form a compact and wearable device. It enables real-time sign language recognition, speech conversion, location tracking, and obstacle detection within a single platform. The system operates using computer vision and deep learning techniques. The camera captures live hand gestures, which are processed using Open CV through steps such as color conversion,noisereduction,contourdetection,andregionof interest extraction. These processed images are then analyzed by a convolutional Neural Network (CNN) model built using TensorFlowto classify gestures accurately.

Recognized gestures are converted into text and then into speech using a text-to-speech engine. The audio output is delivered through a speaker, allowing effective communication between sign language users and others.

Thesystemalsoprovidesnavigationassistance.The GPS module supplies real-time location information, while ultrasonicsensorsdetectnearbyobstacles.Whenanobjectis identified within a set range, an audio alert is generated to enhanceusersafetyduringmovement.Overall,theproposed systemiscost-effective,portable,andscalable,asitrelieson widely available hardware and open-source software. By combining image processing, deep learning, speech synthesis, and navigation features, the Smart Assistive Glasses offer a comprehensive solution to improve accessibility,independence,andqualityoflifefor disabilities individuals.

Key Words: Assistive Technology, Sign Language Recognition,CNN,ComputerVision,RaspberryPi,Open CV,Text-to-Speech,ObstacleDetection.

1. INTRODUCTION

Assistivetechnologyplaysanimportantrolein modernengineeringbyimprovingthelivesofindividuals withdisabilities.Amongdifferentchallenges,hearingand speech impairments create major difficulties in communicationandsocialinteraction.Communicationis essentialforexpressingthoughts,emotions,andneeds; however, individuals who are deaf or unable to speak often depend on sign language, which is not widely understood by the general public. This gap limits interaction in areas such as education, employment, healthcare,anddailyactivities.

Signlanguageisastructuredformofexpression thatuseshandmovements,facialexpressions,andbody gestures. While it is effective within its community, limited awareness among others restricts smooth communication.Asaresult,manyindividualsexperience isolationandrelyonintermediariesforinteraction.

FIGNO:1PROTOTYPESETUP

Recent advancements in embedded systems, computer vision, and artificial intelligence provide opportunities to address these issues. Computer vision enablesmachinestointerpretvisualdatasuchasgestures, whiledeeplearningmodelsimproveaccuracyinrecognizing

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

patterns. In addition, text-to-speech technology converts recognized information into audio output, allowing interactionwithnon-signlanguageusers.

Mobility is another key factor affecting independence.Navigatingunfamiliarenvironmentscanbe challengingwithoutproperassistance.Technologiessuchas GlobalPositioningSystem(GPS)modulessupportlocation tracking, while ultrasonic sensors help in detecting obstacles.However,mostexistingsolutionsfocusoneither communicationornavigation,ratherthancombiningboth functionalities.

Toovercometheselimitations,theSmartAssistive Glasses system is developed as an integrated solution. It combinesgesturerecognition,speechgeneration,navigation assistance,andobstacledetectionwithinasinglewearable device. By using Raspberry Pi and machine learning techniques,thesystemprovidesreal-timeperformanceina compactandcost-effectiveform.

This approach demonstrates how embedded systemsandartificialintelligencecanbeappliedtoenhance accessibility, promote independence, and improve the qualityoflifeforpersonswithdisabilities

2. PROBLEM IDENTIFICATION

Individualswithhearingandspeechimpairments facesignificantchallengesineverydaycommunicationdue to the absence of a common interaction medium with the general population. Sign language serves as an effective methodwithinitscommunity,butlimitedawarenessamong others creates barriers in places such as hospitals, educational institutions, workplaces, and public environments. This often leads to misunderstandings, reducedopportunities,anddependenceoninterpreters.

Existing assistive solutions provide only partial support. Many applications depend on smartphones or continuousinternetconnectivity,whichaffectsreliabilityin real-timesituations.

Someapproachesfocusonlyongesturerecognition and display text without converting it into speech, while othersgeneratespeechbutlackaccuratereal-timegesture detection.

Mobility assistance also remains fragmented. NavigationtoolssuchasGPStrackersandobstacledetection devicesoperateindependently,requiringuserstomanage multiple systems. This increases complexity and reduces convenience,especiallyforwearableapplications.

Additionally, several existing models suffer from limitations such as high computational requirements, low portability, and reduced performance in dynamic environments. Factors like poor lighting conditions, backgroundnoise,andhardwareconstraintsfurtherimpact accuracyandusability.

Therefore, there is a need for a unified, costeffective, and portable solution that integrates communication and navigation features into a single platform. Such a system should provide real-time gesture recognition,speechoutput,locationtracking,andobstacle detectiontoenhanceindependenceandimproveoveralluser experience

3.LITERATURE REVIEW

3.1

Introduction

Assistive technologies focus on improving communicationandmobilityforindividualswithdisabilities. Hearing and speech impairments create barriers in interaction, especially when sign language is not widely understood. In addition, safe navigation in unfamiliar environmentsremainsachallenge.Recentadvancementsin computer vision, artificial intelligence, and embedded systems have enabled the development of intelligent solutions to address these issues. This section reviews existing approaches and highlights their strengths and limitations.

3.2 Evolution of Sign Language Recognition Systems

Recognitionsystemshaveprogressedfromsensorbasedapproachestovision-basedmethods.Earlytechniques used data gloves with embedded sensors to capture hand movements accurately, but they lacked comfort and portability.

Later, camera-based systems using image processingtechniquessuchassegmentation,thresholding, andcontourdetectionimprovedusability.However,these methods were sensitive to lighting and background conditions.

The introduction of deep learning, especially Convolutional Neural Networks (CNNs), significantly improvedperformancebyautomaticallyextractingfeatures from images. These models provide higher accuracy but requirelargedatasetsandcomputationalresources.

3.3 Assistive Communication Technologies

Communication-focused systems aim to convert gestures intomeaningfuloutputs.Initialdesignsgeneratedonlytext, which limited real-time interaction. To improve usability, text-to-speech (TTS) technology was introduced, enabling conversionoftextintovoiceoutput.

Modernsystemscombinegesturerecognitionwith speech synthesis, allowing effective communication with non-signlanguageusers.Despitetheseimprovements,many solutions still face challenges such as processing delays, dependencyoninternetconnectivity,andlimitedreal-time performance

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

3.4 Navigation and Mobility Assistance Systems

Mobilitysupporttechnologiesenhancesafetyand independence. GPS modules are widely used for outdoor navigation, providing real-time location data. However, signallimitationsreduceaccuracyinindoorenvironments. Ultrasonic sensors are commonly applied for obstacle detectionbymeasuring distanceusingsoundwaves.These sensors are simple and cost-effective but have restricted rangeandmaybeaffectedbyenvironmentalconditions.

Wearableassistivedevicessuchassmartcanesand navigation aids have been developed, yet most lack integration with communication systems, limiting their effectiveness.

3.5 Review of Existing Systems

Several research studies have explored different aspects of assistive technology. Vision-based systems achieved real-time gesture recognition but required high computational power. Image processing techniques improved detection accuracy but were affected by environmentalvariations.

Machine learning approaches enhanced classification, while deep learning models such as CNNs provided better accuracy and robustness. Some implementationsusedembeddedplatformslikeRaspberry Pitodevelopcost-effectivesolutions.Inparallel,navigation systemsusingGPSandultrasonicsensorsweredesignedfor mobilityassistance.

However, most solutions focus on specific functionalitiesratherthanprovidingacompletesystem.

3.6Research Gap

Existing technologies lack integration of communication and navigation features into a single platform.Manysystemsrequiremultipledevices,increasing complexityandreducingusability.Limitationssuchashigh cost, reduced portability, and performance issues in realtime environments further restrict their practical application.Therefore,thereisaneedforaunified,compact, and efficient system that combines gesture recognition, speech conversion, location tracking, and obstacle detection. Such a solution can enhance accessibility, improve independence, and provide better support for personswithdisabilities.

4.EXISTING SYSTEM

4.1

Sensor-Based Gesture Recognition Systems

Earlyassistivesolutionsusedwearabledevicessuch as data gloves with embedded sensors to capture finger movements and hand orientation. These approaches provided accurate gesture detection but were expensive, uncomfortable,andunsuitableforlong-termusage.Theneed

forphysicalcontactdevicesalsoreduceduserconvenience andportability.

4.2 Vision-Based Gesture Recognition Systems

Later developments introduced camera-based methods using computer vision techniques. These systems process images through steps such as segmentation,thresholding,contourdetection,andregion extraction to identify hand gestures. Although this approach improved usability and eliminated wearable sensors,performancewasaffectedbylightingconditions, backgroundnoise,andenvironmentalvariations.

4.3 Deep Learning-Based Recognition Systems

Recentadvancementsutilizemachinelearningand deep learning models, especially Convolutional Neural Networks(CNNs), for accurate gesture classification. These methods automatically extract features from images and provide better accuracy compared to traditional techniques. However, they require large datasets and high computational power, making real-time execution on embeddeddeviceschallenging

4.4 Navigation and Communication Assistance Systems

ExistingassistivetechnologiesformobilityuseGPSmodules for location tracking and ultrasonic sensors for obstacle detection. Communication systems often convert gestures into text, while some include text-to-speech features. However, these functionalities are typically developed independently

5.METHODOLOGY/SYSTEM

5.1SYSTEM ARCHITECTURE

DESIGN

5.1.1 Over all architecture of the system

NO:5.1.1ARCHITECTUREOFSYSTEM

FIG

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

The smart assistive glasses designed as a an integrated embedded solution that combines gesture recognition speech generation, navigation, and obstacle detectionwithinasingleplatform.ARaspberryPiservesas thecentralcontroller,managingallcomponentsandsystem operations

Initially,hardwareelementssuchasthecamera,GPS module,ultrasonicsensor,andspeakerareinitialized.After setup, the camera continuously captures hand gestures. These images undergo prepossessing steps including grayscale conversion, noise filtering, and threshold to improveinputquality.

Following prepossessing, the Region of Interest (ROI)isextractedtoisolatethehandfromthebackground, whichenhancesdetectionaccuracyandreducesunnecessary computation. The extracted data is then provided to a Convolutional Neural Network (CNN),whichclassifiesthe gestureandgeneratesthecorrespondingoutput.

The identified gesture is converted into text and furthertransformedintospeechusingatext-to-speech(TTS) engine. The resulting audio is played through a speaker, enablingcommunicationbetweentheuserandothers.

At the same time, navigation and safety features operate alongside gesture processing. The GPS module continuouslyprovideslocationdetails,whiletheultrasonic sensor detects nearby obstacles. When an object is found withinapredefineddistance,anaudioalertisgeneratedto notifytheuser.

All modules function concurrently, ensuring realtimeperformance.

This parallel operation allows seamless communication and navigation support. The compact and wearabledesignmakesthesystemefficientandpracticalfor real-worlduse.

5.1.2 Description of integrated modules

Thesystemconsistsofmultiplemodulesthatwork together to provide a complete assistive solution. The gesturerecognitionmodulecaptureshandmovementsusing aPiCamera,processestheimages,andclassifiesthemusing a CNN model for accurate identification. The speech conversionmoduletransformsrecognizedgesturesintotext andthenintovoiceusingatext-to-speechengine,enabling effectivecommunication.ThenavigationmoduleusesaGPS unit to provide real-time location information, supporting movementinoutdoorenvironments.Theobstacledetection moduleusesanultrasonicsensortoidentifynearbyobjects and generates audio alerts when obstacles are detected withinaspecificrange,ensuringsafety.

All modulesare interconnectedandcontrolledby theRaspberryPi,enablingsmoothdataflowandreal-time operation. This integration improves efficiency, reduces complexity, and provides a compact assistive system. The systemcapturesgestures,classifiesthemusingaCNN,and

convertsthemintospeech.Italsoprovideslocationtracking through GPS and detects obstacles using sensors. All functions are controlled by the Raspberry Pi for real-time operation.

5.2.

BLOCK DIAGRAM

NO:5.2BLOCKDIAGRAM

5.3 DATA SETS

FIGNO:5.3DATASETS

FIG

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

5.4 RESULTS

5.4.1 Gesture Recognition Result

FIGNO:5.4.1GESTURERECOGNITIONRESULT

The gesture recognition module successfully identified hand gestures using the CNN model with good accuracy under normal conditions. Minor variations in lightingslightlyaffectedperformance.

5.4.2 Speech Output Result

The system converted recognized text into clear speech using a text-to-speech engine. The output was understandable with minimal delay, ensuring smooth communication.

5.4.3 Navigation Result

FIGNO:5.4.3NAVIGATIONRESULT

The GPS module provided real-time location trackingwithacceptableaccuracyinoutdoorenvironments. Initialsignalacquisitionrequiredsometime

5.4.4 Obstacle Detection Result

NO:5.4.4OBSTACLEDETECTIONRESULTS

Theultrasonicsensordetectedobstacleseffectively beyond a certain range and generated timely alerts. Very closeobjectswerelessaccuratelydetected.

5.4.5 System Performance Analysis

Thesystemshowedreliablereal-timeperformancewith efficient integration of all modules. Accuracy and responsetimeweresatisfactoryforpracticaluse.

5.4.6 Discussion

Theresultsdemonstratethatthesystemprovides effective communication and navigation support. With further improvements, it can achieve higher accuracy and widerapplicability

6. CONCLUSION

Thedevelopedsystemprovidesaneffectivesolution for assisting individuals with hearing and speech impairments by integrating gesture recognition, speech conversion,navigation,andobstacledetectionintoasingle wearableplatform.

The gesture recognition module, based on a CNN model, achieves reliable performance with improved accuracythroughimagepreprocessing.Recognizedgestures aresuccessfullyconvertedintospeechusingatext-to-speech engine,withenhancedaudioclaritythroughasoundcard.

The navigation module offers real-time location tracking using GPS, while the ultrasonic sensor ensures safety by detecting obstacles and generating alerts. The systemdemonstratesefficientreal-timeoperationwithall modulesfunctioningsimultaneously.

Althoughcertainlimitationsexist,suchassensitivity to environmental conditions and sensor constraints, the system performs effectively within its scope. Overall, the

FIG

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

solution is cost-effective, portable, and user-friendly, improvingcommunication,safety,andindependentmobility.

7. REFERENCES

1) T. Starner and A. Pentland, “Real-time American Sign Language recognition from video using hidden Markov models,” IEEE Trans. Pattern Analysis and Machine Intelligence,vol.20,no.12,pp.1371–1375,Dec.1998.

2) S. Ong and S. Ranganath, “Automatic sign language analysis:Asurveyandthefuturebeyondlexicalmeaning,” IEEETrans.PatternAnalysisandMachineIntelligence,vol. 27,no.6,pp.873–891,Jun.2005.

3) Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature,vol.521,no.7553,pp.436–444,2015.

4) A. Krizhevsky, I. Sutskever, and G. Hinton, “ImageNet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems, pp. 1097–1105,2012.

5) P. Molchanov et al., “Hand gesture recognition with 3D convolutional neural networks,” in Proc. IEEE CVPR Workshops,2015,pp.1–7.

6)S.MitraandT.Acharya,“Gesturerecognition:Asurvey,” IEEETrans.Systems,Man,andCybernetics,vol.37,no.3,pp. 311–324,May2007.

7) R. Kadous, “Machine recognition of Auslan signs using PowerGloves,” IEEE Trans. Neural Systems and Rehabilitation Engineering, vol. 14, no. 1, pp. 103– 108, 2006.

8) D. Kelly et al., “A review of vision-based gesture recognitionmethods,”IEEEAccess,vol.7,pp.95552–95571, 2019.

9)M.Everinghametal.,“ThePASCALVisualObjectClasses Challenge,”InternationalJournalofComputerVision,vol.88, no.2,pp.303–338,2010.

10)G.Bradski,“TheOpenCVlibrary,”Dr.Dobb’sJournalof SoftwareTools,2000.-62

11)J.Redmonetal.,“YouOnlyLookOnce:Unified,real-time objectdetection,”inProc.IEEECVPR,2016,pp.779–788.

12) A. Howard et al., “MobileNets: Efficient convolutional neural networks for mobile vision applications,” arXiv:1704.04861,2017.

13) A. S. Huang et al., “Visual odometry and mapping for autonomous navigation,” IEEE Robotics & Automation Magazine,vol.18,no.4,pp.70–79,2011.

14) J. Borenstein and Y. Koren, “Obstacle avoidance with ultrasonic sensors,” IEEE Journal of Robotics and Automation,vol.4,no.2,pp.213–218,1988.

15)E.KaplanandC.Hegarty,UnderstandingGPS:Principles andApplications,2nded.,ArtechHouse,2005.

16) M. S. Grewal, L. R. Weill, and A. P. Andrews, “Global positioning systems, inertial navigation, and integration,” IEEEAerospaceandElectronicSystemsMagazine,vol. 16, no.6,pp.3–10,2001

17) R. Want, “An introduction to IoT,” IEEE Pervasive Computing,vol.14,no.1,pp.2–4,2015.

18) S. K. Nayar and T. Mitsunaga, “High dynamic range imaging: Spatially varying pixel exposures,” IEEE Trans. PatternAnalysisandMachineIntelligence,vol.22,no.9,pp. 996–1006,2000.

19)H.Saketal.,“Longshort-termmemorybasedrecurrent neuralnetworkarchitecturesforspeechrecognition,”IEEE SignalProcessingMagazine,vol.29,no.6,pp.82–97,2014.

20) M. Abadi et al., “TensorFlow: Large-scale machine learningonheterogeneoussystems,”IEEESoftware,vol.33, no.6,pp.96–104,2016