Application of Object Detection on Snakes

Figure 5: Classifying animals in an image using LabelImg [3]

After all of the data (images) are labeled LabelImg will transform the data into a TFRecord, which trains the data in TensorFlow. In TensorFlow, a label map must be created, which marks all of the classifications in the TFRecords. Such as if the classified data included different objects. Next, the TFRecords in converted into a trained model with TensorFlow and applied to the object detection API to start detecting objects in real-time. Alternatively, pre-trained data can be inputted into the TensorFlow API to be used to create a real-time object detection software.

Object detection is a professional practice utilized in the field of computer science. Moreover, it is used by computer science researchers and programmers to solve real-world problems. A significant dilemma is a venomous snakebite and diagnosis by medical professionals in developing countries. Such as “globally, at least 421,000 envenoming and 20,000 deaths occur each year due to snakebite [and] in Malaysia, according to a five-year review of snakebite patients admitted to a tertiary university hospital, there were 260 cases of snakebites reported, and 52.9 % of them were bitten by unidentified species” [4]. Furthermore, a solution utilizing object detection has been implemented by analyzing the prominent features of a snake and using machine learning techniques to aid medical professionals in detecting the species of snake that bit the patient. This allows the medical professional to apply the correct anti-venom to the patient and adequately diagnose their injury. Computer science researchers utilize different neural networks and databases to create object detecting software to resolve this problem. Furthermore, this article will go over the use of TensorFlow Lite, and Efficient Nets which use different image classification algorithms, which can be implemented to the object detecting software to detect snake species more accurately or efficiently. These different algorithms were implemented in the object detecting software created by the computer science researchers because “that snake species identification is a challenging task, primarily because of the high diversity of snake species, high intra-class variance, and low inter-class variance” [5].

Object Detection of Snakes Utilizing TensorFlow Lite

TensorFlow Lite allows for TensorFlow models to work on mobile devices. Thus, allowing machine learning algorithms to run on mobile devices by using low binary size and latency (delay before a transfer of data begins) [3]. Because of TensorFlow Lite, real-time snake detecting software can be implemented into mobile devices. This can be, for example, an app that a medical professional dealing with patients with snake bites could download from the Google Play Store or Apple App Store. To make the software of detecting snakes “run on smartphones, embedded devices, or even edge devices smoothly and could detect in real-time, the trained neural network [must be converted] from float data format to in format (float32 to unit8). This conversion has subtle, or even no accuracy lost with less storage size and faster inference time” [1]. The reason researchers use TensorFlow Lite as the algorithms that detect snakes is

because of the speed and less storage utilized so the software can run smoothly on more miniature computing-powered devices such as Android smartphones or iPhones.

Mobile Application and TensorFlow Lite Software

An example of the practice of object detection applied to approach further to solve the issue of venomous snakes related injuries, and proper diagnosis of the injury can be seen with my Real-Time Snake Detector Mobile Application, named SeeSnake. I began this idea with a preliminary project by modeling and then developing the software with machine learning using: TensorFlow Lite API, Google Colab notebooks, GDrive, 500 image snake data set, and python programming language. SeeSnake Preliminary Project Overview

To begin this main project l, I started by building a custom snake real-time object detector software as a preliminary project.

Preliminary Project Steps, I: 1. Gathered 500 snake images in various lightings, and outdoor environments. 2. Performed image classification with snake dataset using LabelImg (Figure 6). 3. Used Google Collab free GPU for training and saved that data into the Google Drive to maintain organization. 4. Analyzed the TensorFlow model from the dataset training. 5. Exported the trained model and used my local machine to test with videos of a scenario. 6. Analyzed software testing for False Negative (snake is present but is not detecting) and True Negative (snake is not present and detecting).

Figure 6: Classifying snakes in image using LabelImg

The data used in the preliminary project was beneficial since I was fortunate to have a close friend that has a passion for taking pictures of outdoor amphibians and reptiles. I was able to receive 75% of my data set from these professional images. Furthermore, the software I created using the TensorFlow object detection API had flaws in some cases but overall was successful in a majority of the cases. This is greatly attributed to the small dataset I worked with but was perfect for the preliminary software. The main purpose now was that I had a fully working directory (software model), as shown below, that I would implement in the main project.

SeeSnake Main Project Technology Overview

The main project would be to now implement the software and model I created in the preliminary project into a mobile application for Android and IOS devices. This would be a real-time snake detector

application, which utilizes the smart phones front camera to detect the snake, which can then alert the user through the application.

Main Project Steps (Figure 7), I: 1. Gathered 10,000 snake images in various lightings, and outdoor environments. 2. Performed object detection with snake dataset using the TensorFlow Lite Object Detection API. 3. Used Amazon paid GPU for training and saved that data into the Amazon data base to maintain organization. 4. Analyzed the TensorFlow model from the dataset training. 5. Exported the trained model and used my local machine to test with videos of a scenario. 6. Analyzed software testing for False Negative (snake is present but is not detecting) and True Negative (snake is not present and detecting). 7. Design the mobile application on Figma 8. Code the mobile application with React Native 9. Connect the software to the mobile application 10. Launch the application on the Apple App Store and the Google Play Store

The SeeSnake app can be downloaded on iPhone devices by downloading the “Expo Go” app on the Apple App Store. Following by opening the camera app on the iPhone and scanning the QR code seen in Figure 8.

Figure 7: Steps taken in main project for SeeSnake

Figure 8: QR code to download SeeSnake App

Snake Object Detection using Efficient Nets

Computer science researchers have developed other machine learning techniques to help make detecting snakes more accurately other than utilizing TensorFlow Lite. “The SnakeCLEF 2020 challenge, which is part of the LifeCLEF research platform”, was a competition in which several teams competed to make the best snake image detecting software [6]. The “University of Applied Sciences and Arts Dortmund (FHDO) Biomedical Computer Science Group (BCSG)” applied Efficient Nets to their machine learning

algorithms for detecting snakes [6]. EfficientNet is a convolutional neural network that uses a compound coefficient to scale the snake image input into several more images input in dimensions and angles alternatives as seen in Figure 9 [4]. This allows for more data to be analyzed by the snake object detecting software. This resulted in the team having one of the most accurate snakes detecting software in the 2020 SnakeCLEF competitions [6].

Figure 9: The EfficientNet technique applied to image detection of snakes [6]