Personalized Driver Alerting System using object detection

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 10 Issue: 10 | Oct 2023

p-ISSN: 2395-0072

www.irjet.net

Personalized Driver Alerting System using object detection John Pius Thayiparampil1, Arun Binoy2, Shijina B3, Alby Alphonsa Joseph4, Athira R Kurup5, Safna Sainudeen6 1-2BTECH UG Students, Department of Computer Science and Engineering, TOMS college of Engineering 3Head of the department, computer science and engineering, TOMS college of Engineering 4-6Assistant professor, computer science and engineering, TOMS college of Engineering 3 APJ Abdul Kalam Technological University, Kerala, India

---------------------------------------------------------------------***---------------------------------------------------------------------

Abstract - The development of intelligent solutions to help

2010/40/EU directive [1]. The collision avoidance system is one of the ITS systems used to address driving safety, i.e., to prevent or lessen auto accidents. There have been numerous approaches to the collision avoidance issues documented.

Here are a few of those mentioned. In [2], an intelligent system for vehicle driving safety warning based on a support vector machine was presented. Two functions—lane departure warning and forward collision warning—were taken into consideration.

drivers prevent potential crashes is crucial given the growing need for road safety. This work introduces a paper “Personalized Driver Alerting System using Object Detection" that aims to improve driving safety by instantly alerting the driver when objects, including people, cars, and obstructions, cross the path of the vehicle. The object detection technology used in the system is google TensorFlow object detection. The system is quite good at spotting possible collision threats because it uses sophisticated object identification techniques to analyses the breadth of the bounding boxes formed around items that are discovered. In doing so, it reduces the likelihood of collisions and improves general traffic safety. This system's object identification algorithm, which can identify a variety of items including as pedestrians, other cars, and other road impediments, is essential to its operation. The system accurately recognizes and classifies items in the surroundings by utilizing neural networks and cutting-edge computer vision technology. Most notably, it evaluates the breadth of the bounding boxes that surround these items. The language used for this software is python programming language. This width analysis, which assesses the possible threat level an object offers to the vehicle, is a crucial part of our system's decisionmaking process. The danger increases with decreasing breadth, alerting the driver with more strong and rapid warnings. The system's warnings are highly configurable and may be customized to each driver's needs. The ability to customize alarm thresholds and notification styles for drivers guarantees a relaxing and customized driving experience.

Alerts can be given by haptic feedback, aural warnings, or visual signals, which increases driver engagement and lowers distraction.

A fuzzy controller was employed in [3] to provide early warning in a collision avoidance system that used a radar to gauge the speed and distance between cars. A vehicle collision warning system with user-defined safety distance based on vehicle-to-vehicle communication was presented in [4]. A fuzzy approach was used to analyse driving style, environmental conditions, vehicle speed, and safety distance in a collision avoidance control technique that was published in [5]. Additionally, two controls were created for speed management: a PID control and a sliding mode control.

Information technologies like as fuzzy logic, support vector machines, and conventional controls are used in the design of collision avoidance systems in the aforementioned methodologies. Since deep learning has significantly improved neural model performance, in this research we apply Google TensorFlow Object recognition API [6], a deep learning-based object recognition technique, to the suggested driving safety warning system. The structure of this document is as follows. The suggested system is explained in Section 2 along with a general flowchart.

Subsequently, Section 3 provides specifics on the three main features of the suggested system. In Section 4, a highway case is examined and a real-world experiment is carried out to support the suggested method. Lastly, Section 5 presents the conclusion.

Key Words: Object Detection, Object identification techniques, Bounding boxes, Object identification algorithm, Computer vision technology, Width analysis, Decisionmaking process, Alarm thresholds.

1.1 PERSONALIZED DRIVER ALERTING SYSTEM

1.INTRODUCTION

The creation of clever solutions to assist drivers in this endeavor has been spurred by the urgent need to improve road safety and avert potential accidents. This abstract presents a response to this request with the paper "Personalized Driver Alerting System using Object Detection."

The principal aim of this study is to considerably improve

In recent times, there has been a growing interest in and research on intelligent transportation systems (ITS). ITS systems are those that use information and communication technology in road transport, as per the European Union's

© 2023, IRJET

|

Impact Factor value: 8.226

|

ISO 9001:2008 Certified Journal

|

Page 818

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 10 Issue: 10 | Oct 2023

p-ISSN: 2395-0072

www.irjet.net

road safety by instantly alerting drivers when things, such as pedestrians, cars, and impediments, come into contact with their path. This system's use of advanced object recognition techniques, which carefully evaluate the bounding box dimensions surrounding discovered items, is essential to its efficacy. This strategy is essential for reducing collision risk and, as a result, improving traffic safety in general. This study aims to provide an in-depth analysis of the "Personalised Driver Alerting System using Object Detection," highlighting its innovative features and potential to enhance road safety

1.2 ENHANCING ROAD INTELLIGENT SOLUTIONS

SAFETY

clever ways to avert possible collisions. This project's main objectives are as follows: B) Improving Road Safety: The main goal is to increase road safety by promptly warning drivers when objects, including people, cars, and impediments, come into their route.

C) Advanced Object Identification: To accurately identify possible collision threats, the system uses advanced object identification techniques to evaluate the bounding box width surrounding identified objects.

THROUGH

D) Reducing Collision Likelihood: The system is essential in lowering the probability of crashes and improving overall traffic safety since it can recognize and categories a variety of objects in the surrounding area, including people, other cars, and roadblocks.

The sophisticated object identification algorithm at the heart of this system can identify a wide variety of items in the surrounding environment, including pedestrians, other cars, and road obstructions. The combination of cutting-edge computer vision technologies and neural networks allows for this high degree of recognition accuracy. Interestingly, one of the most important aspects of the system's decisionmaking process is how wide the bounding boxes are that contain these things. The algorithm determines the possible hazard that each object poses to the car by examining the width of these bounding boxes; smaller widths indicate greater danger and initiate louder, more urgent alarms.

E)

Neural Networks and Computer Vision: The system's efficacy is derived from its reliance on state-of-the-art technologies, such as computer vision and neural networks, for precise object recognition and classification.

F) Threat Assessment via Width Analysis: One of the most important parts of the system's decisionmaking process is the width analysis of the bounding boxes, which determines the possible threat level that objects may pose. Greater danger is indicated by narrower box widths, which prompts louder and earlier alerts.

G) Customized Alerts: The system provides highly customizable alerts that are able to be tailored to the individual requirements of each driver. This personalization guarantees a comfortable and customized driving experience.

Fig -1: Object Detection used in road safety

H) Various Notification Styles: To increase driver engagement and reduce distractions, alerts can be displayed visually, audibly, or through haptic feedback.

This technology is unique in that it can be tailored to meet the needs and preferences of specific drivers. The system generates warnings that may be customised to meet the individual needs of each driver. This includes the ability to change the notification styles and alarm thresholds. This adaptability enables a variety of alert choices, including as haptic feedback, audio warnings, and visual signals, which improve driver involvement while reducing distractions. It also guarantees a customised and stress-free driving experience.

2. LITERATURE REVIEW With the growing need for effective measures to reduce traffic accidents and save lives, it is imperative to develop intelligent solutions targeted at improving road safety and averting possible crashes. The abstract in this instance includes a research named "Personalized Driver Alerting System using Object Detection," which offers a novel strategy for enhancing road safety. In the article, a system that immediately alerts drivers when items like cars, pedestrians, or barriers enter their route is proposed. The progress of an

2. OBJECTIVE A) The Personalized Driver Alerting System with Object Detection seeks to improve traffic safety by creating

© 2023, IRJET

|

Impact Factor value: 8.226

|

ISO 9001:2008 Certified Journal

|

Page 819

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 10 Issue: 10 | Oct 2023

p-ISSN: 2395-0072

www.irjet.net

expansion of research is reported in Design a Support Vector Machine-based Intelligent System for Vehicle Driving Safety Warning [2]. The result is a gadget that is fitted inside a longhaul bus and used on a regular basis. The current system uses the support vector machine (SVM) as the classifier and combines the lane departure warning (LDW) and forward collision warning (FCW) functions. The vehicle anti-collision warning system simulation model is constructed using fuzzy inference rules of certain parameters, accordingly, and a simulation test is carried out in The Study of Vehicle's Anticollision Early Warning System Based on Fuzzy Control [3].

is capable of precisely identifying and categorizing a wide range of nearby objects, such as other cars, pedestrians, and roadblocks. Specifically, the system measures these bounding boxes' width, which is an important factor in determining how dangerous an object might be for the car.

This width analysis is used by the system to decide how serious of a warning the driver should receive. Additionally, the system's customizable alerting features—which include visual signals, aural cautions, and haptic feedback—provide a customized, distraction-free driving experience that improves driver involvement and overall safety. The principal aim of this project is to create a clever solution that improves driving safety by promptly notifying the driver when items, such as people, cars, and impediments, come into the route of the truck. The project's goals are to increase overall traffic safety and lessen the chance of collisions.

In Calculating the separations between other vehicles and the collision alert [8] Sensors are used by forward collision warning (FCW) systems to identify objects in front of moving vehicles. In Is Google TensorFlow Object Detection API the easiest way to implement image recognition [6] The API has been trained on the COCO dataset (Common Objects in Context). Designed of Improved Vehicle Collision Warning System Based on V2V Communication [4] Vehicle-to-vehicle (V2V) communication allows vehicles to send position and speed data to each other. The goal is to prevent accidents.

Longitudinal Control Strategy of Collision Avoidance Warning System for Intelligent Vehicle Considering Drivers and Environmental Factors [5] provides the driver a safety and comfortable driving experience, by transferring early warning signals to avoid collision while detecting potential danger. In context, Microsoft Common Objects [7] has A large-scale dataset for object recognition, segmentation, and captioning is called COCO.

3.2 DATA COLLECTION AND MODEL SELECTION Comprehensive data gathering is a fundamental part of the project in order to create the "Personalized Driver Alerting System using Object Detection". A large dataset of actual driving situations is used by the system to train and optimize the Google TensorFlow object detection model. The algorithm's resilience and flexibility are guaranteed by the variety of locations, road conditions, and traffic situations included in this dataset. Vehicles outfitted with an array of sensors, including cameras and LIDAR, are used in data collection to gather a variety of visual and spatial data from the environment. The system's sensors keep an eye on traffic signals, road signs, and driver behavior all at once to build a comprehensive picture of the driving environment.

3. METHODOLOGY At least four essential components make up the suggested system described: frontal car detection, human identification, obstacle detection, safety factor computation, frontal car distance estimation, and driving state discrimination.

The Google TensorFlow Object Detection (GTOD) API is used in the suggested system to detect cars in real time. The Microsoft Common Objects in Context (COCO) dataset [7], which has 300,000 photos for 90 objects, is used to train the GTOD API. Table 1 lists the five models that can be chosen from the GTOD API based on accuracy and speed.

3.1 GOOGLE TENSORFLOW OBJECT DETECTION

Table -1: Models in the GTOD API

The creation of clever ways to improve traffic safety is crucial, particularly when it comes to averting possible collisions. The study "Personalized Driver Alerting System using Object Detection" addresses this issue by using Google TensorFlow object identification, a state-of-the-art technique. By instantly alerting drivers when objects, such as pedestrians, other vehicles, or impediments, enter their route, this technology significantly contributes to increased driving safety. The system's ability to detect possible collision hazards is attributed to the application of sophisticated object recognition algorithms that carefully examine the extent of bounding boxes surrounding objects that are recognized. By doing this, it greatly reduces the possibility of accidents and advances traffic safety in general.

An essential component of this system's functionality is its object identification algorithm, which is driven by neural networks and cutting-edge computer vision technologies. It

© 2023, IRJET

|

Impact Factor value: 8.226

Due to its efficiency, the "ssd_mobilenet_v1_coco" model is employed in the Pr0opopsed system.

Furthermore, because the suggested system is intended to be used in a typical highway driving scenario, only three

|

ISO 9001:2008 Certified Journal

|

Page 820

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 10 Issue: 10 | Oct 2023

p-ISSN: 2395-0072

www.irjet.net

categories—people, obstacles, cars, buses, and trucks—are taken into account.

3.2 FEATUTRE EXTRACTION IDENTIFICATION

AND

J)

OBJECT

False Positive Mitigation: Create algorithms or postprocessing methods to reduce false positive warnings, particularly in difficult situations like bad weather or dim lighting.

K) Model Ensemble: To increase overall accuracy and resilience, think about combining predictions from several object detection models using ensemble approaches.

A) Data Augmentation: To artificially boost the dataset's diversity, use augmentation techniques. Rotation, flipping, and brightness modifications are a few techniques that can improve the model's generalization.

L) Continuous Learning: Put in place a system that allows the model to adjust itself over time to new objects, changing road conditions, and possible safety hazards. This may entail using online learning resources or retraining on a regular basis.

B) Backbone Selection: Select the backbone architecture that best suits the model of object detection. Reset, Mobile Net, or Efficient Net are popular options. The decision should strike a balance between computing efficiency and accuracy.

C) Anchor Box Tuning: Adjust the dimensions of the anchor box so that they correspond to the scales and aspect ratios of typical roadside objects. This optimization can increase the precision of object detection.

D) Transfer Learning: Use transfer learning to your advantage by starting the model with weights that have already been well trained on large-scale datasets (COCO). Transfer learning speeds up the model's acquisition of features pertinent to your particular task.

E) Object tracking: Use object tracking strategies to keep objects' identities consistent between frames.

This may contribute to a decrease in false alarms and an increase in alert consistency.

Fig -2: Anchor box tuning using labelimg

F) Calibration and Camera Correction: To guarantee precise object size estimation, take into consideration camera distortions and calibration problems. To achieve accurate bounding box analysis, proper camera calibration is essential.

G) Model Optimization: To guarantee real-time performance in the setting of a moving vehicle, optimize the model for distribution on edge devices.

The model's size and processing requirements can be decreased by employing strategies like model quantization.

Fig -3: created an xml when the image got labeled

H) Object Categorization: In addition to detection, put in place a system for classifying things into appropriate groups, such as cars, pedestrians, or road signs. This may make the alerts more educational.

I)

The "Personalized Driver Alerting System using Object Detection" project makes use of technologies like LabelImg, which are specialized software. An essential part of the research, Labeling makes it easier to annotate and name objects in the large dataset that is required to train the object detection model. With the aid of this program, project researchers can make annotated photographs by drawing bounding boxes around a variety of things of interest,

Machine Learning Explain ability: To increase the transparency of the system's decision-making process, apply explain ability techniques. Both system debugging and user trust depend on this.

© 2023, IRJET

|

Impact Factor value: 8.226

|

ISO 9001:2008 Certified Journal

|

Page 821

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 10 Issue: 10 | Oct 2023

p-ISSN: 2395-0072

www.irjet.net

including as cars, people, and impediments on the road. The ground truth data needed to train the object detection algorithm inside the Google TensorFlow framework is provided by these annotated photos. The flexibility and ease of use of LabelImg's interface are vital in expediting the data labelling procedure, guaranteeing the precision and caliber of the annotated dataset, and eventually bolstering the efficacy of the driver alerting system.

It is anticipated that the suggested system will function in typical highway driving conditions. For greater efficiency, just three relevant categories from the COCO dataset are taken into account. They are classified as humans, obstacles, vehicles (truck, car, bus, motorcycle etc.). In this paper, the three object categories shall collectively be referred to as "cars." The GTOD handles object detection using the three categories that have been chosen. We extend the GTOD API to include the Python enumerate function in order to retrieve the coordinates of cars that have been detected. The relevant box width (w) for every car identified can be determined using those coordinates. This is represented as w=xmax-xmin, where xmax and xmin stand for the box's leftand right-side x-coordinates, respectively, for the identified automobile under consideration.

3.3 Alerting Mechanism Figure in this portion displays the general flowchart of the suggested system. The following outlines the relevant steps.

Read a frame first. Second, use the GTOD API to identify frontend automobiles. Third, look up each detected object's confidence level (CL). Proceed to the next step if CL is greater than 50%; if not, return to the first step. Mark boxes for objects that are detected, fourth. Fifth, find each identified object's box width (w). Calculate the safety factor s=(1-w)^4 in the sixth step. Seventh, if s > 0.5, a safe state is identified; display boxes and repeat step 1 for an additional frame; if S<=0.5, display 'WARNING' and the distance for any automobiles that have been spotted. To differentiate between a dangerous and a warning state in this instance, more investigation is required.

At least three fundamental components make up the proposed system outlined frontal object recognition, frontal car distance estimation and safety factor computation, and driving state differentiation. The ensuing subsections include the pertinent details.

Fig -5: detecting car Safety factor s is computed by w of each detected automobile. A greater s is deemed to be a safer condition in the suggested system. It has been noted that a greater w corresponds to a closer distance from the frontal car that was identified, and vice versa. As a result, s rather than w should be proportionate to 1-w. Moreover, take note of the non-linear relationship between s and frontal vehicle distance df.

Figure illustrates the plot for Table 1 and confirms the quasilinear relationship between s and df. The frontal vehicle distance in the suggested system is estimated using the corresponding linear fitting, df' = 29.9-1.39. The third row of Table 1 provides the estimated distance for various s. It can be disregarded because, in this instance, the state is secure and causes no harm. According to Table, for s=0.5, df=13,8m is shown. A driver has enough time to handle difficulties at this distance. As a result, the threshold of s=0.5 is established to distinguish between driving states. Information about state discrimination is provided.

Fig -4: The flowchart for the proposed system

© 2023, IRJET

|

Impact Factor value: 8.226

|

ISO 9001:2008 Certified Journal

|

Page 822

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 10 Issue: 10 | Oct 2023

p-ISSN: 2395-0072

www.irjet.net

matching safety factor of 0.5, which is on the threshold.

Therefore, in this instance, another check should be made. It is determined that the frontal car is in the same lane because the box's centre of x-coordinate, cx, is within the interval [0.3,0.7]. As a result, the suggested system views this situation as risky. shows the digit 14 m, rounded.

Furthermore, the box is labelled in magenta, and the screen displays the word "WARNING" in red.

Fig -6: relationship between s and df Three driving states—the safe state, the warning state, and the risky state—are identified by safety factors. The threshold in the proposed system to distinguish between driving states is chosen at s=0.5. A state is deemed safe when s¿0.5, or the frontal car distance, surpasses 13.8 meters.

Fig -8: dangerous stated object detected

Either a warning state or a dangerous condition is the outcome for the situation s¡=0.5. Whether or not the frontal automobile is in the same lane determines the outcome. The following criterion is applied in order to differentiate this. If the center x- coordinate of a detected automobile, cx, is found inside the pre-defined range of the normalized frame width [0.3,0.7], it will be examined.

It is possible to modify the software for the "Personalized Driver Alerting System using Object Detection" mentioned in the above abstract so that it functions on mobile devices.

Modern mobile devices have strong CPUs and sophisticated capabilities, which makes them ideal for executing sophisticated object detection algorithms.

4. RESULT A screenshot taken during a highway test of the suggested system. The widths of two identified frontal automobiles are computed by the boxes. There is a greater than 0.5 safety factor for both. As a result, the circumstances are regarded as safe. With an estimated distance of 22.5 meters, the car in the same lane has a safety factor of s=0.8. The boxes are yellow-green in this state.

Fig -8: software operate in a mobile phone The programmed can be optimized and turned into a mobile application to make it functional on mobile devices. The camera on the mobile device could be used by this software as a real-time input source to record the environment around the car.

Fig -7: safe stated object identified The second example is the screen grab that is displayed in Figure. A dangerous or warning situation is indicated by the

© 2023, IRJET

|

Impact Factor value: 8.226

|

ISO 9001:2008 Certified Journal

|

Page 823

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 10 Issue: 10 | Oct 2023

p-ISSN: 2395-0072

www.irjet.net

Furthermore, the LabelImg software is essential to the research since it makes the process of labelling data simpler, which helps to produce a high-quality annotated dataset that is utilized to train the object detection model. This software combination promises safer and more customized driving experiences for all users, marking a major advancement in the ongoing efforts to employ technology to improve road safety and prevent possible crashes. In this study, we have suggested a real-time car detection and safety alert system based on the Google TensorFlow Object Detection (GTOD) API.The suggested system's two main components were automobile detection and driving state discrimination.

During the car detection phase, frontal autos were boxed and their widths were calculated. The safety factor was then computed using the box's width.

Fig -9: accuracy of object detection in night To provide effective real-time processing, the object detection model—which was first constructed using Google TensorFlow—can be modified and optimized for mobile technology.

Based on the safety factor, three states were considered for the driving state discrimination. Furthermore, an approximate distance was given for each safety element. The proposed solution was tested in real-world highway driving conditions. The results of the experiment have shown that it is feasible in the given circumstances.

ACKNOWLEDGEMENT The task has been successfully completed, and I am very appreciative to God Almighty for his grace and blessings, without which this study would not have been possible.

REFERENCES Fig -10: object detector detect the every person A highly developed and intelligent system created with the primary goal of locating and reducing potential road accident risks. It has advanced object identification capabilities, with a special emphasis on identifying pedestrians and other items that can endanger a moving vehicle. The programmed analyses and recognizes items in its range of view by utilizing sophisticated object identification techniques and Google TensorFlow object detection technologies. In addition to pedestrians, other cars and obstacles on the road are also included, all of which are essential components in preventing accidents.

[1]

Directive 2010/40/EU of the European Parliament and of the Council of 7 July 2010. Available at: eurlex.europa.eu

[2]

C.-C. Lin, C.-W. Lin, D.-C. Huang, and Y.-H. Chen, “Design a Support Vec- tor Machine-based Intelligent System for Vehicle Driving Safety Warning,” IEEE Conference on Intelligent Transportation Systems, pp. 938-943, Beijing, China, 2008.

[3]

F. Cheng, D. Zhu, and Z. Xu, “The Study of Vehicle’s Anitcollision Early Warning System Based on Fuzzy Control,” International Conference on Computer, Mechatronics, Control and Electronic Engineering, pp. 275-277, Changchun, China, 2010.

[4]

Y.R. Liu, J.X. Guo, ‘Designed of Improved Vehicle Collision Warning System Based on V2V Communication,” IEEE International Conference on Electronics information and Emergency Communication, pp. 95-98, Beijing, China, 2018.

[5]

Y. Zhao, X. Xiang, R. Zhang, L. Guo, and Z. Wang, “Longitudinal Control Strat- egy of Collision Avoidance Warning System for Intelligent Vehicle Considering Drivers and Environmental Factors,” IEEE Intelligent Vehicles Symposium, pp. 31-36, Suzhou, China, 2018.

3. CONCLUSIONS Annotation tools like LabelImg and Google TensorFlow object detection are essential to the creation and implementation of the "Personalized Driver Alerting System using Object Detection." The fundamental technology of the system is Google TensorFlow object detection, which allows it to precisely identify and categories things in the path of the car, improving road safety by lowering the chance of crashes. Each user will have a customized and distractionfree driving experience thanks to the powerful object identification technology and customizable alerting features.

© 2023, IRJET

|

Impact Factor value: 8.226

|

ISO 9001:2008 Certified Journal

|

Page 824

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 10 Issue: 10 | Oct 2023

p-ISSN: 2395-0072

www.irjet.net

[6]

Priya Dwivedi. (2017). Is Google Tensorflow Object Detection API the eas- iest way to implement image recognition? https://towardsdatascience.com/isgoogle-tensorflow- object-detection-api-the-easiestway-to-implement- image- recognition-a8bd1f500ea0 SIGACCESS Conf.

Comput.

Access.

(2019). https://doi.org/10.1145/3308561.

[7]

T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, P. Dollar, “Microsoft common objects in context,” arXiv: 1405.0312v3 [cs.CV], 21 Feb 2015.

[8]

Harrison Kinsley. (2017). Determining other vehicle distances and collision warning. Available at: https://pythonprogramming.net/detecting-distancesself- driving-car/

© 2023, IRJET

|

Impact Factor value: 8.226

|

ISO 9001:2008 Certified Journal

|

Page 825