Deep Learning for Large-Scale Traffic-Sign Detection and Recognition

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 08 Issue: 11 | Nov 2021

p-ISSN: 2395-0072

www.irjet.net

Deep Learning for Large-Scale Traffic-Sign Detection and Recognition Divya Munot Maharashtra, India ---------------------------------------------------------------------***-------------------------------------------------------------------

Abstract- The first self-driving cars are on their way to becoming a reality. As highways get more congested, drivers and autonomous vehicles will have a tougher difficulty viewing all of the traffic signs. It is hard to discount the danger of missing critical cues that will set the stage for disaster. A camera-based traffic sign recognition system was created to assist drivers and self-driving automobiles in overcoming this difficulty. After being trained on a significant amount of data, including synthetic traffic signs and photos from street views, the proposed multi-task convolutional neural network (CNN) refines and classifies the data to their precise classifications.

Post processing analyses all of the input frames before to making a recognition judgment. The suggested system classified traffic indicators using the CNN algorithm. The efficacy of the proposed methodology has been demonstrated through experimentation.

increase road safety, but also helps drivers navigate unfamiliar or difficult roads. To create a system that uses a built-in camera to snap real-time images of traffic signs, identify the symbol's meaning, and notify the driver by voice command.

3. LITERATURE SURVEY Hengliang Luo, Yi Yang, Bei Tong, Fuchao Wu, and Bin Fan,.

“Traffic Sign Recognition Using a Multi-Task Convolutional Neural Network”, [1] Despite the fact that traffic sign identification has been examined extensively for a long period of time, the majority of prior study has focused on traffic signs using symbols. This paper introduces a novel data-driven strategy for recognising all forms of traffic signals in video sequences captured by a car camera, including those with symbols and those with text. The system is separated into three stages: extracting areas of interest (ROIs) from traffic signs, refining and classifying the ROIs, and post-processing. To extract traffic sign ROIs from each frame, the most stable extremal regions on grey and modified RGB channels are used. The proposed multi-task convolutional neural network trains on a massive quantity of data, including synthetic traffic signs and photos tagged from street views, and then refines and classifies them precisely.

Finally, post processing combines the data from all frames in order to generate a recognition judgement. The utility of the proposed technology has been shown experimentally.

Key Words: CNN, Machine Learning, pre-processing, Classification autonomous driving, traffic sign, deep learning, detection.

1. INTRODUCTION To find an issue with an advanced driver assistance system (ADAS), you must know how it works. Knowing what's going on inside the system will help you figure out why something isn't operating properly. This prevents you from spending money on things that aren't broken. Examine a traffic sign recognition system in greater detail. As soon as a new road sign is detected, the car's sign recognition technology alerts the driver. They're usually shown on a monitor in the instrument cluster.. Road signs are typically "seen" using a forward-looking camera situated behind the windshield.

There are automobiles that have a separate forward-facing camera for this system, while others use the same ADAS camera for lane departure warning and other tasks. In the proposed system, traffic signs are identified using deep learning, which yields Excellent results. In deep learning, the CNN method analyses and separates various aspects in an input image. There is far less pre-processing required than with other classification techniques. Filters are handdesigned by people with the necessary expertise, following simple techniques. Automated driving and driver assistance systems rely heavily on traffic sign detection.

Sungho Kimh and Soon Kwon,.”Improvement of traffic sign recognition by accurate ROI refinement “,[2] The accuracy influence of the detected traffic sign region on traffic sign recognition (TSR) is presented in this research, and an improved TSR approach based on accurate traffic sign region extraction is provided. The traditional HOG-based traffic sign detection (TSD) has poor accuracy in terms of localisation.

Inaccurate traffic sign regions have a direct impact on TSR performance. The effect of localization error is studied using speed limit signage for the specific TSR. The increased TSR performance is validated by improved colour channel based traffic sign extraction in the TSR learning and testing step.

D. M. Filatov, K. V. Ignatiev, E. V. Serykh , “Neural Network System of Traffic Signs Recognition”, [3] This work proposes a method for real-time detection and recognition of traffic signals while accounting for differences in illumination and distance. A Raspberry 2 single-board computer and a Hama AC-150 camera were used to accomplish the proposed approach. A colour filter with morphological operators and a Canny edge detector are used to determine the location of traffic signs, and a multilayer perceptron neural network is utilised to determine the sign kind. Five distinct traffic signs were used to train and evaluate an algorithm. As a result,

2. PROBLEM STATEMENT To ensure a smooth traffic flow free of roadblocks and disasters, road signs are required. Road symbols are pictorial representations of numerous bits of information that the driver must comprehend. Drivers frequently ignore traffic signs in front of their vehicles, which can result in serious accidents. To create technology that not only seeks to

© 2021, IRJET

|

Impact Factor value: 7.529

|

ISO 9001:2008 Certified Journal

|

Page 392

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 08 Issue: 11 | Nov 2021

p-ISSN: 2395-0072

www.irjet.net

experiments were successfully accomplished. The created system is impervious to light variations and is capable of recognising traffic signals with a diameter of 20 cm from a distance of 1.5–2 metres.

in the segmentation section, and the system extracts the geometry-based characteristic of the traffic sign in the extraction section. We then give these geometry-based elements of traffic signs to the classification to be classed and predicted, and then it detects the traffic sign and transforms it to a voice warning based on that assumption in classification, where we use our CNN algorithm to classify and forecast.

Catur Supriyanto, Ardytha Luthfiarta, Junta Zeniarja..,“An Unsupervised Approach for Traffic Sign Recognition Based on Bag-of-Visual-Words”,[4] There are numerous suggestions for improving riding safety. The Advanced Driver Assistance System (ADAS) is a system that assists the driver in maintaining a higher level of safety. The goal of ADAS is to help and direct drivers in order to improve traffic safety. One of the most crucial aspects of ADAS is road signs recognition. A traffic sign is a warning sign that is placed on the side or above the road to give the motorist with detailed driving information. We present an unsupervised strategy for traffic sign identification based on a bag of-visual-word model in this paper. For detection and recognition, an unsupervised technique does not require label data or a training process. When we have a lot of data that isn't labelled, it helps. The data used in our experiment came from the Deutsche Sign Recognition Standard (GTSRB). It's a public dataset for recognising traffic signs. The results reveal that using a large number of visual words allows the proposed method to build high-quality clusters with high accuracy. Although the study's accuracy is still relatively poor. The reason for this is that each image only generates 34 keypoints. The modest image size has an impact on the small number of keypoints.

Yi Yang, Hengliang Luo, Huarong Xu, and Fuchao Wu., “Towards Real-Time Traffic Sign Detection and Classification”,[5] Road signs reading is crucial in driver assistance and intelligent autonomous vehicles. Along with its recognition rate, its true performance is quite desirable.

This work focuses on real-time traffic sign recognition, which requires rapidly determining the type of traffic sign that appears in which region of an input image. To do this, we propose a detection module that is 20 times faster than the fastest detection module currently available. Our detection module is based on the extraction and classification of proposed traffic signs via a colour probability model and a colour HOG. The observed signs are then categorised further using a deep neural network into subcategories in each superclass. Experiments on German and Chinese roadways demonstrate that our detection and classification algorithms outperform state-of-the-art techniques while consuming significantly less processing power.

Figure 1. System Architecture 5. ALGORITHM CNN (Convolution Neural Network) Convolutional Neural Networks (CNNs) are neural networks that specialize in image and video recognition. Image recognition, object detection, and segmentation are among of the most common image analysis tasks that CNN is employed for.

There are four types of layers in Convolutional Neural Networks: 1) Convolutional Layer: Each input neuron in a traditional neural network is linked to the next hidden layer. Only a small portion of each layer's neurons connect to the hidden neurons in CNN.

4. PROPOSED SYSTEM

2) Pooling Layer: The pooling technique is used to reduce the dimensionality of the data map. A number of activation and pooling layers will be present in the CNN's hidden layer.

This system may be programmed to accept input in the form of a photograph. We know we'll be doing image processing on the system, so we'll apply our CNN algorithm in all four image processing modules: preprocessing, segmentation, feature extraction, and classification. So, the image is first input as an image, followed by preprocessing the image dataset (cleaning the dataset and RGB to Gray to Binary conversion). The image is then segmented into small pixels

© 2021, IRJET

|

Impact Factor value: 7.529

3) Flatten: The process of flattening data into a onedimensional array for usage in the following layer is known as flattening. We flatten the output of the convolutional layers to create a single long feature vector.

|

ISO 9001:2008 Certified Journal

|

Page 393

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 08 Issue: 11 | Nov 2021

p-ISSN: 2395-0072

www.irjet.net

4) Fully-Connected layer: The network's final layers are known as Fully Connected Layers. The output of the final Pooling or Convolutional Layer is flattened and fed into the completely connected layer as the input to the completely connected layer.

Rojo-Álvarez, “Traffic sign segmentation and classification using statistical learning methods,” Neurocomputing, vol.

153, pp. 286–299, Apr. 2015.

[5] M. Haloi. (2015). “A novel pLSA based traffic signs classification system.”

[Online].

Available: https://arxiv.org/abs/1503.06643

How to work CNN 1.

Build a small convolutional neural network as defined in the architecture below.

2.

Select images to train the convolutional neural network.

3.

Extraction of feature filters/feature maps.

4.

Implementation of the convolutional layer.

5.

Apply the ReLu Activation function on the convolutional layer to convert all negative values to zero.

6.

Then apply max pooling on convolutional layers.

7.

Next Flatten, This layer used for convert 2D matrix into 1D array.

8.

Make a fully connected layer

9.

Then input an image into CNN to predict the image content

[6] Y. Zhu, C. Zhang, D. Zhou, X. Wang, X. Bai, and W. Liu, “Traffic sign detection and recognition using fully convolutional network guided proposals,” Neurocomputing, vol. 214, pp. 758–766, Nov. 2016.

[7] R. Timofte, V. Prisacariu, L. Van Gool, and I. Reid, “Combining traffic sign detection with 3D tracking towards better driver assistance,” in Emerging Topics in Computer Vision and Its Applications. Singapore: World Scientific, 2011, pp. 425–446. doi: 10.1142/8103.

[8] A. Mogelmose, “Visual analysis in traffic & reidentification,” Ph.D. dissertation, Fac. Eng. Sci., Aalborg Univ., Aalborg, Denmark, 2015.

[9] J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, “Man vs.

Computer: Benchmarking machine learning algorithms for traffic sign recognition,” Neural Netw., vol. 32, pp. 323–332, Aug. 2012.

[10] S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing, and C. Igel, “Detection of traffic signs in real-world images: The German traffic sign detection benchmark,” in Proc. IJCNN, Aug. 2013, pp. 1–8.

10. Back propagation to calculate the error rate

[11] A. Mogelmose, M. M. Trivedi, and T. B. Moeslund, “Vision-based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey,” IEEE Trans. Intell. Transp. Syst., vol. 13, no. 4, pp.

1484–1497, Dec. 2012.

11. Then Create CNN model.

6. CONCLUSION A typical traffic sign recognition system consists of localization and classification algorithms that are executed in the same order. Localization is the process of determining the position of the traffic sign within the frame, while classification is the process of matching the sign to a trained CNN model of traffic sign classes.

[12] F. Zaklouta and B. Stanciulescu, “Real-time traffic-sign recognition using tree classifiers,” IEEE Trans. Intell. Transp.

Syst., vol. 13, no. 4, pp. 1507–1514, Apr. 2012. [13] Z. Zhu, D. Liang, S. Zhang, X. Huang, B. Li, and S. Hu, “Traffic sign detection and classification in the wild,” in Proc. CVPR, Jun. 2016, pp. 2110–2118.

REFERENCES [1] V. Balali, A. A. Rad, and M. Golparvar-Fard, “Detection, classification, and mapping of U.S. traffic signs using Google street view images for roadway inventory management,” Vis. Eng., vol. 3, no. 1, p. 15, 2015.

[14] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask RCNN,” in Proc. Int. Conf. Comput. Vis., Oct. 2017, pp. 2980– 2988.

[15] S. B. Wali, M. A. Hannan, A. Hussain, and S. A. Samad, “Comparative survey on traffic sign detection and recognition: A review,” Przegl¸ad Elektrotechniczny, vol. 1, no. 12, pp. 40–44, 2015.

[2] K. C. P. Wang, Z. Hou, and W. Gong, “Automated road sign inventory system based on stereo vision and tracking,” Comput.-Aided Civil Infrastruct. Eng., vol. 25, no. 6, pp. 468– 477, 2010.

[16] J. Jin, K. Fu, and C. Zhang, “Traffic sign recognition with hinge loss trained convolutional neural networks,” IEEE Trans. Intell. Transp. Syst., vol. 15, no. 5, pp. 1991–2000, Oct. 2014.

[3] V. Balali and M. Golparvar-Fard, “Evaluation of multiclass traffic sign detection and classification methods for U.S. roadway asset inventory management,” J. Comput. Civil Eng., vol. 30, no. 2, 2016, Art. no. 04015022. [4] J. M. LilloCastellano, I. Mora-Jiménez, C. Figuera-Pozuelo, and J. L.

© 2021, IRJET

|

Impact Factor value: 7.529

|

ISO 9001:2008 Certified Journal

|

Page 394

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 08 Issue: 11 | Nov 2021

p-ISSN: 2395-0072

www.irjet.net

[17] V. Vukoti´c, J. Krapac, and S. Šegvi´c, “Convolutional neural networks for croatian traffic signs recognition,” in Proc. CCVW, 2014, pp. 15–20.

[18] D. Cire¸san, U. Meier, J. Masci, and J. Schmidhuber, “A committee of neural networks for traffic sign classification,” in Proc. IJCNN, Jul./Aug. 2011, pp. 1918–1921.

[19] Y. Zeng, X. Xu, Y. Fang, and K. Zhao, “Traffic sign recognition using deep convolutional networks and extreme learning machine,” in Proc. IScIDE, vol. 9242, 2015, pp. 272– 280.

[20] M. Haloi. (2015). “Traffic sign classification using deep inception based convolutional networks.” [Online]. Available: https://arxiv. org/pdf/1511.02992.pdf [21] J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, “The German traffic sign recognition benchmark: A multi-class classification competition,” in Proc. IEEE Int. Joint Conf.

Neural Netw., Aug. 2011, pp. 1453–1460.

[22] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, “OverFeat: Integrated recognition, localization and detection using convolutional networks,” in Proc. Int. Conf. Learn. Represent., 2014, pp. 1–16.

[23] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. Int. Conf. Learn. Represent., 2015, pp. 1–14.

[24] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. CVPR, Jun. 2016, pp.

770–778.

[25] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real time object detection with region proposal networks,” in Proc. NIPS, 2015, pp. 91–99.

[26] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proc. Comput. Vis. Pattern Recognit., Jul. 2017, pp. 936– 944.

© 2021, IRJET

|

Impact Factor value: 7.529

|

ISO 9001:2008 Certified Journal

|

Page 395