Performance Analysis of CNN Trained for Traffic Sign Recognition on Cadence Tensilica Vision P6 DSP

8

X

https://doi.org/10.22214/ijraset.2020.31927

October 2020

International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.429 Volume 8 Issue X Oct 2020- Available at www.ijraset.com

Performance Analysis of CNN Trained for Traffic Sign Recognition on Cadence Tensilica Vision P6 DSP Ninad Patil1, Nachiket Patil2, Mangesh Malgi3 1

Department of Electronics and Telecommunication, College of Engineering, Pune 2 Integrated Device Technology, USA 3 Department of Electrical Engineering, Indian Institute of Technology Bombay

Abstract: Automatic Driver Assistance System (ADAS) technology is currently in an embryonic stage. Many tech-companies multinationals, as well as start-ups, are working on developing a truly autonomous vehicle that will guarantee the safety and security of the passengers as well as other vehicles, pedestrians on roads, and roadside structures such as traffic signal poles, traffic signposts, and other structures. But still, these autonomous vehicles have not been implemented on a large scale for regular use on roads currently. These autonomous vehicles perform many different object detection/recognition tasks. Examples of which include but are not limited to traffic sign recognition, lane detection, pedestrian detection. Usually, such tasks are performed by the person driving the vehicle. The main goal of such autonomous systems should be to perform these tasks in real-time. With the advent of deep learning, these object recognition tasks can be performed with very high accuracy. Usually, these neural networks are implemented on a hardware device that does all the work of computation. Many different hardware choices are available from different vendors suiting the client's needs. Examples of hardware on which these neural networks are implemented include CPU, digital signal processor, graphics processing unit, and FPGA. The underlying processor forms an important part of an ADAS on which these CNN are implemented and processes the incoming frames for object detection/recognition task from a camera in real-time. Real-time processing is necessary to take appropriate actions/decisions depending on the logic embedded. Hence knowing the performance of the neural network (in terms of frames processed per second) on the underlying hardware is a major factor in deciding the various hardware options available from different vendors, which CNN model to implement, whether the CNN model is suitable to implement on the underlying hardware depending upon the system specifications and requirement. In this paper, we trained a convolutional neural network for traffic sign recognition task using Nvidia DIGITS web-based software and analyzed the performance of the CNN in terms of fps on Cadence's Tensilica Vision P6 DSP as underlying hardware using Cadence's proprietary software's such as Xtensa Neural Network Compiler and Xtensa Xplorer.

Keywords: Traffic Sign Recognition (TSR), Convolutional Neural Networks (CNN, ConvNets), Multiply and Accumulate (MACs), frames per second (fps), Digital Signal Processor (DSP), Graphics Processing Unit (GPU), Nvidia DIGITS, Xtensa Neural Network Compiler (XNNC), Xtensa Xplorer, Xtensa Processor Generator (XPG), Vector Floating Point Unit (VFPU) I. INTRODUCTION Cadence provides SoC designers with the world's first configurable and extensible processor, fully supported by automatic hardware and software generation. Cadence Tensilica Xtensa processors enable SoC designers to add performance, elasticity, and endurance to their designs through software programmability, as well as differentiation through processor implementations custom-made for their application. Cadence Tensilica Vision DSPs are customizable and are based on Xtensa architecture. Various Vision DSP are available P5, P6, Q6, etc. As these DSPs are designed to perform in an optimized way for image processing specific operations, they provide high energy efficiency as well as high performance. The architecture of the vision DSP's is tailored to implement embedded vision tasks with optimal performance and power efficiency (classification/detection tasks). The Tensilica Vision DSPs are synthesizable processors. Features that make this processor favorable for image processing tasks are as follows: 1) Specially tailored Instruction sets for performing Image processing tasks (Add, Sub, Compare, Multiplication, Divide).

2) With new instructions, increased math throughput, and other enhancements.

3) For AI applications, the Vision P6 DSP boosts performance by up to 4X with quadruple the available MAC horsepower, which is a major computation block for AI applications.

4) Compared to commercially available GPUs, the Vision P6 DSP will achieve twice the frame rate at much lower power consumption on a typical AI implementation.

ŠIJRASET: All Rights are Reserved

494

International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.429 Volume 8 Issue X Oct 2020- Available at www.ijraset.com 5) For a wide range of other key vision functions, such as convolution, FIR filters, and matrix multiplies, the Vision P6 DSP increases performance by up to 2X with its improved 8-bit and 16-bit arithmetic.

6) Optional VFPU with single-precision 32-bit and/or half-precision 16-bit floating-point support offers performance and flexibility for porting existing GPU code.

Fig. 1 Various features of Vision DSPs that enable image processing tasks to be performed in an optimized way A. Xtensa Neural Network Compiler XNNC is the tool for quantizing a pre-trained floating-point neural network model for Tensilica DSPs. XNNC helps to optimize the CNN by efficiently converting the floating-point weights to an optimized fixed-point weight without losing on the accuracy. XNNC supports both standard and custom CNN layers. Major CNN architectures are supported by XNNC. CNN's are optimized according to the underlying hardware selected among the various vision DSP for implementation. XNNC quantizes the input model and generates the C implementation code for this neural network. In this case, we selected the Vision P6 DSP hardware for implementing.

1) XNNC Input a) Floating-point weight file (.caffemodel) b) Calibration and Validation dataset 2) XNNC Output a) Optimized CNN code for target Xtensa DSP. b) Other supporting files The XNNC undergoes a multi-stage process to generate an optimized version of the CNN. In this process, it analyses the layer activations of the floating-point network and evaluates quantization profiles (min/max ranges of the outputs from each layer) across a range of experimental distributions observed in the calibration and validation image sets supplied by the user. Based on this evaluation, scaling factors for fixed-point conversion of each layer are determined. The XNNC then takes these quantization parameters and generates an optimized, fixed-point neural network. The optimized code along with other supporting software files needed for the application is provided in the form of a workspace. This workspace file is generated with a .xws extension. This workspace file can then be imported into Xtensa Xplorer to get a performance report.

B. Xtensa Xplorer IDE The Xtensa Xplorer IDE serves as the graphical user interface (GUI) for the entire design process. From the Xtensa Xplorer IDE, designers with existing application software can profile their application, identify hot spots, decide on configuration options, add instructions and execution units to optimize performance, and then generate a new processor. This puts flexibility directly into the hands of the designer with a tool that integrates software development, processor optimization, and multi-processor SoC architecture in one IDE. Xtensa Xplorer IDE is a part of the Cadence Tensilica Xtensa SDK. The Tensilica Xtensa SDK provides a comprehensive collection of code generation and analysis tools that speeds the application software development process. Xtensa Xplorer IDE can be configured for analyzing the performance on the Vision P6 DSP, by importing the P6 DSP core from XPG. Once the Xtensa IDE is configured for Vision P6 DSP then the performance report of the neural network can be generated by importing the previously generated workspace from XNNC.

ŠIJRASET: All Rights are Reserved

495

International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.429 Volume 8 Issue X Oct 2020- Available at www.ijraset.com The Xtensa Processor Generator creates a complete hardware design with matching software tools, including a mature, world-class compiler, a cycle-accurate SystemC-compatible ISS, and the full industry-standard GNU toolchain.

Fig. 2 Architecture of Vision P6 DSP

Fig. 3 Energy consumption comparison between CPU, GPU, and Vision DSP

ŠIJRASET: All Rights are Reserved

496

International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.429 Volume 8 Issue X Oct 2020- Available at www.ijraset.com As compared to CPU and GPU which are usually used to implement a CNN, Vision DSPs consume very less amount of energy/frame. This is due to the optimizations made in the architecture of the DSP to perform image processing tasks and also the features mentioned above that makes these image processing tasks easier to perform.

Fig. 4 Flow diagram for XPG II. LITERATURE REVIEW CNN is used for object detection/recognition tasks owing to their accuracy in detecting/recognizing objects in the images and the speed with which they perform these tasks. CNN consists of different layers that are connected to one after the other in a cascaded manner. Some of the common types of layers are convolutional layer, fully connected layer, loss layer, pooling layer. The number of layers of each type and the order in which these layers are connected is what defines a CNN architecture. Some of the common types of CNN architectures are ResNet, GoogleNet, AlexNet, MobileNet. The architecture that was chosen for the TSR task was AlexNet. A. AlexNet Architecture Input to AlexNet is an RGB image of size 256*256. However, 227*227 random crops are generated from the 256*256 image and fed to the first layer of the AlexNet. AlexNet architecture consists of 8 layers, 5 convolutional layers, and 3 fully connected layers in the end. ReLU, Multi-GPU training, and overlapping max-pooling were introduced. ReLU non-linearity is given by f(x) = max (0, x). Some of the convolution layers are followed by ReLU, Batch Normalization, and Max Pooling in order. The final fully connected layer feeds to the SoftMax classifier. This classifier classifies the image into 1 of the 43 classes of traffic signs.

Fig. 5 AlexNet Architecture

ŠIJRASET: All Rights are Reserved

497

International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.429 Volume 8 Issue X Oct 2020- Available at www.ijraset.com III. METHODOLOGY A. Training of the CNN for Traffic Sign Recognition Task AlexNet neural network was trained using the Caffe framework for german traffic sign recognition task using Nvidia DIGITS. Caffe is a deep learning framework developed by the Berkeley Vision and Learning Centre at the University of California, Berkeley. The network definition format is completely different from other frameworks. It was developed by keeping speed in mind. Once the model is trained, a .caffemodel file is generated. This file consisted of floating-point weights/parameters. Other files which are required for training are:- 1) train_val.prototxt – this file is used to define the network architecture, define the path to a binary proto file, path to training and validation dataset 2) solver.prototxt – used to specify gradient descent parameters, learning rate, number of iterations, step-size, base learning rate, and other information required for training 3) mean.binaryproto – It has the mean value of images over a complete dataset for each channel that needs to be subtracted from each image. In total, the model was trained on 43 different classes of traffic signs. NVIDIA DIGITS provides a GUI for the training of a neural network. Transfer learning was used for training the model for TSR task. DIGITS take care of generating and modifying other training files as mentioned before. Trained caffe model was downloaded from caffe model zoo. This model was then uploaded onto the DIGITS software along with the traffic sign dataset consisting of 43 image classes in jpeg format. DIGITS can be used for performing pre-processing steps on a dataset, selecting/uploading a pre-trained model for transfer learning, train a neural network from scratch, and also provides various visualizations (in the form of graphs/charts) before, during, and after training. Training was completed and a .caffemodel file was generated along with other prototxt and binaryproto file. This .caffemodel file generated is fed to the XNNC software for analysis purpose along with TSR dataset.

B. Analysis of the CNN on XNNC All the hardware and software pre-requisites for the XNNC were installed and then the XNNC package was extracted for installing.

The (. caffemodel) file was given input to the XNNC along with the calibration and validation dataset. The image format was ppm, and the image resolution was 227*227*3. Both these datasets contained 11 images of each class. In total there were 11*43 = 473 images in each dataset. There are a total of 2 stages in the compilation: - 1) Analysis Stage 2) Optimization Stage. The Analysis stage is run first, using a combination of calibration and validation images, and the network definition. The Analysis stage maps the network layers to their XI-CNN library equivalents and empirically evaluates the activations of each layer to determine appropriate quantization parameters for the model weights and library functions.

C. 1) a) b) c) 2) 3) 4)

Analysis Stage Options Quantization Accuracy Levels: Level 0 (default – better performance, lesser detection quality) – This level was selected for this task Level 1 (Balanced) Level 2 (better detection quality) Adjustable thresholds for min/max selection and outlier removal for quantization calibration Layer wise manual modification of min/max values for quantization Control of input/output data types for custom layers

D. Optimization Stage Options The Optimization stage is run second, where it takes the quantized network and fixed-point parameters, and generates the optimized code for the target Xtensa processor. The Optimization stage determines how local memory is partitioned, the storage formats of data and coefficients, and the DMA tiling strategy for the network.

1) Layer wise declaration of data storage formats (WHD, DWH, etc.)

2) Memory Configuration Options 3) Use default configuration from the target DSP's core parameters 4) Treat dual local RAMs as contiguous or non-contiguous 5) Adjust the size of memory reserved for stack and statically allocated data 6) Batching (process a series of images to help reduce memory bandwidth) When the compilation process is complete, optimized code is generated for the target Xtensa processor, and a summary of the expected detection quality of the quantized, fixed-point network. XNNC strictly requires the Caffe model of a neural network to be used for generating the vision code. We can use a TensorFlow model.

©IJRASET: All Rights are Reserved

498

International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.429 Volume 8 Issue X Oct 2020- Available at www.ijraset.com Using this model will require one more additional step to be performed, converting the TensorFlow model into the Caffe model. If we use Caffe, then we can bypass this conversion step as shown in the image below.

Fig. 6 XNNC Compilation Stages and User Control IV. RESULTS AND DISCUSSIONS A. Training Results As the images are not of the same size, resizing needs to be done. There are multiple resizing options available in Nvidia DIGITS. Examples are squash, fill (random noise), crop, half-crop. After using each option and examining the training loss result for a predetermined no of epochs, it was found out that the fill option gave the best results. As shown in the image below training loss is very low and the validation accuracy is around 97%.

Fig. 7 Accuracy achieved in squash mode

ŠIJRASET: All Rights are Reserved

499

International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.429 Volume 8 Issue X Oct 2020- Available at www.ijraset.com

Fig. 8 Accuracy achieved in fill mode B. Performance Results The performance report is obtained from Xtensa Xplorer software. The image below represents the result. It can be concluded from the "Layer Name" column that the neural network used for the TSR task is AlexNet. There are 5 convolutional layers followed by 3 fully connected layers (ip6, ip7, ip8) followed by the SoftMax layer. Convolutional layers 1 and 2 are followed by batch normalization and max pooling. The convolutional layers 3,4,5 are cascaded, followed by max-pooling in the end. Most of the MAC operations are performed in the convolutional layers and fully connected layers themselves. The no of MACs per cycle and the total number of MACs for each layer is mentioned. DMA wait cycles – No of cycles the request must wait to access the memory directly via a system bus. DMA Queue Size - the number of DMA requests queue to reserve in local memory.

Fig. 9 Performance Report

ŠIJRASET: All Rights are Reserved

500

International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.429 Volume 8 Issue X Oct 2020- Available at www.ijraset.com V. CONCLUSIONS From the simulation which was performed on Xtensa Xplorer it was found out that the Cadence Tensilica Vision P6 DSP on which the AlexNet was implemented to recognize traffic signs processed the input video frames at a rate of 116.54 fps at a processor clock speed of 1000 MHz (1GHz). Major portions of the total MAC operations are done in convolutional and fully connected layers only.

The total no of MAC operations required for processing one frame is 720 Million. Thus 8580570 total cycles are required for processing one frame. Processor clock speed is 1000 MHz i.e. 1000000000 clock cycles in 1 second. Thus 1000000000/8580570 comes out around 116.54 fps. This indicates it can process one frame in 8.58ms. This time is sufficient to take a logical decision based on the traffic sign category. The performance of the CNN can further be increased by using the different options present in the Xtensa Xplorer software to implement the CNN more optimally.

VI. FUTURE WORKS The performance of the CNN can further be increased by using the different options present in the Xtensa Xplorer software to implement the CNN more optimally. Current results assumes ideal conditions that no latency and error will occur while recognising the traffic sign. The overall system can be made more robust for different weather conditions and other scenarios. RTL code corresponding to the Vision P6 DSP can also be generated from the XPG. This RTL code can then be dumped onto an FPGA. We can perform power and area analysis. The average power consumed while the CNN is running on the FPGA and the area occupied in terms of logic elements of the FPGA. Further application-specific integrated circuit chips that perform the TSR task can also be fabricated if they are produced on a large scale on a multi-project wafer lab to divide the cost among different clients.

VII. ACKNOWLEDGMENT I would like to extend my gratitude towards Mr. Sarang Shelke and Mr. Abhishek Belkonikar of Cadence who helped me to complete this project and solved my queries regarding the Cadence XNNC and Xtensa Xplorer software. I would also like to thank my college faculty Mrs. Vanita Agarwal and lab staff for all the guidance and help provided in completing this project.

REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14]

Jerry Wei. (2019) AlexNet: The Architecture that challenged CNNs. [Online]. Available: https://towardsdatascience.com/alexnet-the-architecture-thatchallenged-cnns-e406d5297951 J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, "The German Traffic Sign Recognition Benchmark: A multi-class classification competition," in Proc.

IEEE International Joint Conference on Neural Network, 2011, p. 1453-1460.

(2019) Vision DSPs for Imaging and Vision. [Online]. Available: https://ip.cadence.com/vision Cao J, Song C, Peng S, Xiao F, and Song S, "Improved Traffic Sign Detection and Recognition Algorithm for Intelligent Vehicles", Sensors, vol 19, pp. 4021, Sept. 2019.

Farag, Wael, "Recognition of traffic signs by convolutional neural nets for self-driving vehicles", International Journal of Knowledge-based and Intelligent Engineering Systems, vol 22, pp. 205-214, Nov. 2018.

Software Development. [Online]. Available: https://ip.cadence.com/swdev C. Rowen. Handling The Hard Data Processing Tasks in SoCs. [Online]. Available: https://ip.cadence.com/knowledgecenter/know-ten/dataplane-design NVIDIA DIGITS. [Online]. Available: https://developer.nvidia.com/digits NVIDIA. (2020) Deep learning DIGITS documentation. [Online]. Available: https://docs.nvidia.com/deeplearning/digits/digits-installation/index.html Sumit Saha. (2018) A Comprehensive Guide to Convolutional Neural Networks – the ELI5 way. [Online]. Available: https://towardsdatascience.com/acomprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53 Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv:1408.5093, 2014.

S. Sheng. (2016) A step by step guide to Caffe. [Online]. Available: https://shengshuyang.github.io/A-step-by-step-guide-to-Caffe.html Xtensa Neural Network Compiler, Cadence, 2018.

S. Lapuschkin. (2019) Model Zoo. [Online]. Available: https://github.com/BVLC/caffe/wiki/Model-Zoo

ŠIJRASET: All Rights are Reserved

501