How to Create a Driverless Team from 0 with No Resources

Disclaimer This is supposed to be a detailed guide on How To Create a Driverless Division In An Established Formula Student Team. The steps described here are not intended to be an absolute truth and Formula Student is about innovating and thus making things different.

However, the author has been in an established FS Driverless team and has led the way to founding another one - from zero DV experience - just like many of the readers of this first introduction to this document.

If you don’t know where to start, then you have come to the right place. Let’s begin.

The author feels this video: https://www.youtube.com/watch?v=ly_ax8w-T7E is all the proof the reader needs in order to trust him on following this guide.

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Week 0 - Preliminaries Goal: For every member on the team to have the right tools to begin working.

Things every member will need: Laptop or computer: where all their work will take place USB stick with at least 4GB (one for all the team also works) (Optional) External Hard Disk Drive (or Solid State Drive) with at least 200GB free space: if they have no / don’t want to take 200GB of free space from their laptop/computer (If using a MacBook) External (USB) keyboard and mouse In order to develop our driverless software stack, it’s important to have the right OS to do it.

Linux-type operating systems are known to be the best for coding and their open-source architecture blends naturally with many tools that will help us along the way.

For the tools we want to use, it’s almost impossible to use them in Windows or MacOS. So this is a mandatory step. From all the Linux distributions, Ubuntu is the most popular and easiest to use for beginners while not compromising performance or utility.

Linux (Ubuntu) Installation alongside Windows/MacOS This step is really, really delicate. The author himself has installed multiple Ubuntus in his lifetime and there’s almost no way to evade little mistakes and we have to learn to deal with them.

What we will do is firstly do a back up of what we have in our laptop. This is a really important and mandatory step too. Especially for MacOS users. If you can get help from computer engineering students or IT professionals in your university, do that. If not, don’t worry about it too much, after the third installation it won’t seem that hard.

Why do we need to install Ubuntu alongside our Windows/MacOS? Why can’t we use a Virtual Machine? Because we will need the maximum performance of the OS.

Virtual Machine emulating Ubuntu inside Windows.

1

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Tutorials online on: Dual booting Ubuntu and Windows/MacOS The best way to dual boot Ubuntu and Windows/MacOS - that is, having both operating systems on the same machine - is to follow these tutorials online below. They have proven useful and are the ones the author has followed when starting Hong Kong’s Red Bird Racing own Driverless Division.

For Windows | https://itsfoss.com/install-ubuntu-1404-dual-boot-mode-windows-8-81-uefi/ This one is a very detailed tutorial and has been proven to work in more than 10 laptops following it (with Windows 8 and 10 on them). Normally, there shouldn’t be many questions when following it. However, there’s one part that is wrong for what we want to do: the part called Approach 2: “You don’t see ‘Install Ubuntu alongside Windows Boot Manager’ option or it is greyed out”.

It’s actually recommended to use Approach 2 (no possibility to erase any Windows files)!

The correct size of partitions Basically what’s wrong with it is the size of the partitions. We should allocate at least 100GB (in the very minimum) for Ubuntu, if we can we will allocate 200-250GB to it. Here are the values you should put on “Size” on the particular partitions if you follow “Approach 2: Something else”.

Mount point: / Description: This is the root partition. All the programs and tools will be installed here.

Particularities: Don’t let it reach 100% of used space, that will hang your computer. Ubuntu warns you if there’s 1GB left. So don’t worry.

New value:

75000 MB Justification: The programs needed are not small (especially if you have GPU) and it’s better to be safe having ~75GB of it than worry if we are running out.

Swap area Description: This is the swap area. It is reserved for when the computer needs more resources when the processes are taking all the available memory.

Particularities: We cannot store files there, it’s just an area for the computer to run smoothly when our programs are demanding.

New value:

16000 MB Justification: Our programs will be demanding, so we want to be safe in this one and do ~16GB.

2

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Mount point: /home Description: This is the home partition. Here we will store our scripts, test data, images and everything else that we will have except for programs and tools.

Particularities: This is the everything-else partition so it will be the bigger one, while keeping a safe size for the other two. You actually don’t need it and you could put all the space here on the root partition. But look below for the benefits.

New value:

Everything else (at least 10000 MB) Justification: Here are the benefits of having a dedicated home partition: https://www.lifewire.com/ do-you-need-home-partition-2202048.

The three partitions defined in the tutorial, with the values changed to the correct ones.

Also, for “Device for boot loader installation” is a matter of if we want to choose between Ubuntu or Windows when we turn on the computer or if we want to always load Windows and then change to Ubuntu (see below how to do it). See this answer here: https://askubuntu.com/questions/219514/where-to-install-bootloader-when-installing-ubuntuas-secondary-os.

As the reader can appreciate, the three new values amount to:

75000 MB + 16000 MB + 10000 MB ·

1 𝐺𝐵 1024 𝑀𝐵

≈ 98.6 GB

So that’s the reason you need (at absolute minimum, again) 100GB for Ubuntu. It’s good to have more than 100GB for /home only because our test data will not be small (~10GB for a single trackdrive with cameras, lidar and IMU readings). Members opting for allocating too little space will have to keep in mind that and they will have to use external hard drives for working with all that data.

Another option is to install Ubuntu on an external Hard Drive (or SSD). This is a real possibility and the author has done it twice, it works perfectly too. However, for practical use it is annoying to have a hard drive always plugged into our laptop (and if the cable gets unplugged, even dangerous for our Ubuntu). Only do this if you really don’t have the free space inside your laptop.

The most common “problem” for Windows (Readers wanting to first boot up on Windows and occasionally going to Ubuntu should also do all these steps!) After following the tutorial, the most common problem that will arise is related to the boot loader. It’s important to know what it is so we don’t panic when weird things happen. Since we are installing 2 operating systems in our computer, the computer has to know which OS has to load everytime we turn it on.

3

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

The appearance of the GNU GRUB boot loader.

So first of all, we must not panic about the horrible appearance of the GNU GRUB boot loader: no, our computer has not been hacked by a russian hacker and we have not installed a virus! It is now your new normality.

The Problem: you see this screen below

GRUB in rescue mode

This is the “I should never have trusted this f***ing tutorial” screen. Before you think that that’s it for your computer, read closely. What we will do first is reboot our computer and repeatedly click our BIOS key the second the computer turns on again. We can try multiple keys and see which one does the trick. Here are the typical ones.

Brand

BIOS Key

Acer

F2 or DEL

ASUS

F2, ESC, or DEL

Dell

F2 or F12

HP

ESC or F10

Lenovo

F2 or Fn+F2

Lenovo (Desktops)

F1

Lenovo (Thinkpads)

Enter+F1

MSI

DEL

Origin

F2

Samsung

F2

4

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Sony

F1, F2, or F3

Toshiba

F2

Courtesy of @jamescalam from Gist Github: https://gist.github.com/jamescalam/9d57d5a442b6f7721c14a5385a1c32bb

It’s very important to reach the BIOS settings, once we are there, we have to search for the BIOS options. It is usually a list of boot options where we should see options like: DVD, USB, UEFI and an option containing Windows. Sometimes it may be called OS boot manager and inside, you will find Windows. If you find Secure Boot enabled, disable it.

Typical BIOS options menu appearances.

When we find the USB option, we will put it on top of the priority list and later the Windows/OS option. Once we have it up there, save settings and quit the BIOS setup.

Reboot the computer after quitting.

After that you should go directly to Windows. Nothing’s lost! Now, we want to see if our Ubuntu is also functional. What we will do is again an advanced startup. Search in the Windows search bar “Advanced Startup” and it should open this window below.

Search for this window and press “Restart now”.

After that you should see the screen below. Press “Use a device”. If everything went well, you should see one option inside saying “Ubuntu” of some sort. There is your Ubuntu.

5

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Press “Use a device” and then find the “Ubuntu” one.

If you don’t see any option containing “Ubuntu”, restart your computer (it will go to Windows, if not, do the BIOS step again), plug again your USB with the Ubuntu Installation files and follow the tutorial again from Step 5: Boot from live Ubuntu USB. It’s safe now to Erase Ubuntu from the new partition and install it again (check what partition you are formatting please!). Again, getting help at this point is welcome!

If the same black screen appears again, boot up again the computer plugging in the USB stick and we will want to “Try Ubuntu” instead of installing it. If it doesn't boot up directly to the USB, do the advanced startup again. When we are in trying Ubuntu, we will use a tool called Boot Repair that automatically helps with this issue, follow this guide for reference: https://www.howtogeek.com/114884/how-to-repair-grub2-when-ubuntu-wont-boot/.

Creating a Desktop Icon to go from Windows to Ubuntu If you can access your Ubuntu, now we will do a cool trick to do the advanced startup with a Desktop icon so it’s easier to go from Windows to Ubuntu.

Follow this quick tutorial: https://www.howtogeek.com/656672/how-to-create-a-shutdown -icon-in-windows-10/. But instead of the text “Shutdown.exe /s /t 00” we will input:

shutdown /r /o /t 0 That means we will be rebooting (/r) the computer with advanced options (/o) with 0 seconds of delay (/t 0). Oh, and after you create the shortcut you can change its icon for a nicer one: follow this tutorial https://www.computerhope.com/issues/ch001820.htm! You can download custom .ico pictures to put as icons.

Changing the icon of our shortcut.

6

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

For MacOS | https://www.maketecheasier.com/install-dual-boot-ubuntu-mac/ Warning: this will only work on MacBooks prior to the M1 one. Not on the M1.

Before going on the tutorial, follow this quick guide on how to turn off Secure Boot on your Mac: https://www.wikihow.com/Turn-Off-Secure-Boot-on-Mac. This will enable dual boots. If not, errors will show up.

Now that you have done this, you can start with the MacOS tutorial. A backup of your files is really mandatory here, it has more steps than the Windows counterpart. If you can do it on a Windows laptop, better. If not, here it is.

It’s really important to follow the same fix for the partitions’ size. Search on the Dual Boot on Windows section of this document the subsection: “The correct size of partitions”.

As the tutorial suggests, you will have to find and install yourself drivers for your laptop keyboard and mouse pad. If not, you will always need an external keyboard and mouse to use Ubuntu. But it’s also an option.

Ubuntu: the basics If you have never heard of Linux (or Ubuntu) before, you should know it’s very terminal reliant. So know the basics commands: https://www.youtube.com/watch?v=IVquJh3DXUA.

Ubuntu and its terminal.

In Ubuntu we normally get the tools we want with the command sudo apt-get install XYZ, being XYZ whichever tool we want. So let’s install two tools we already need:

git - To download external code, save backups of our code and have version control. curl - To download files from the internet from the command line.

To download them, just input sudo apt-get install and git or curl on the terminal.

7

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

ROS: the libraries to construct our Driverless pipeline We will now install ROS (Robotics Operating System), a big program that basically is a pack of C++ and Python libraries to work on robotics tasks, such as a Driverless car. Follow the official tutorial: http://wiki.ros.org/noetic/Installation/Ubuntu to install ROS Noetic and make sure you install the ros-noetic-desktop-full when picking which one you like to install.

In the following weeks we will learn to work with ROS and why it is so useful for our Driverless pipeline. For now, we don’t do anything with it yet.

FSSIM: the simulator made in Zurich Now that we have Ubuntu and ROS, it’s time to put our skills to test and try to install the simulator that we will use to develop our Vehicle Control algorithms. This simulator is brilliant work by Juraj Kabzan from the legendary AMZ team down in Zurich and has been a centerpiece for many teams to start working on their Driverless pipelines.

However, the simulator was made back in 2019 and we need an upgraded version that works with ROS Noetic and Python 3, that’s why we will clone Barcelona’s modified version found in: https://github.com/Huguet57/fssim-2021.

Follow the steps on the “How to Run It in your Workspace“ section of the website to install it.

After waiting for it to load, does it work? Congratulations! If not, it seems like there’s work for you to do! You can kill any program in Ubuntu by pressing Ctrl + C.

8

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Weeks 1-3 - Primitive Control & Object Detection Goal: Be able to move the car in the simulator and detect objects given an image.

Things at least one member will need: Laptop with GPU (Recommended) GPU being Kepler and newer (ie compute capability 3.0 and above), example models: Pascal, Kepler, Maxwell, Tegra K1 or Tegra X1 - it will bring faster speed when training a neural network Things every member will need: Laptop or computer with the Preliminaries from Week 0 done A smartphone/digital camera to take pictures of cones This weeks will be focused on making the first steps to creating the Control and Perception subsections. This division will prove to be very useful when going further into the algorithms needed since they will require dedicated teams to work on both parts. The division is also really useful because both subteams can work independently from each other. Let’s see what we mean when talking about Perception and Control: ●

Control: subteam dedicated to given the position of the cones, define a path and send the corresponding commands to the actuators to follow it.

●

Perception: subteam dedicated to read, analyse and interpret the data received from the sensors (cameras, LiDAR, IMUs…) to bring reality to a simulator-type environment so Control can apply its algorithms.

So from this definition, we see that Control needs Perception in a practical sense but we can substitute it with the simulator. The goal of Perception is to reach the stability and information-completeness of the simulator.

In this week, it is not encouraged to apply this division yet. That way every member of the team can see both parts and its basics. This will not only help for making the decision to which subteam they want to focus on, but also will help knowing what the other subteam is about so the communication is more fluid.

9

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

ROS basics: moving our car in the simulator It’s time to talk about ROS. It will be a central part of our Driverless pipeline and it’s good to know the details before thinking about any algorithm we want to use. In this tutorial we will learn the basics while trying to make our car move in the FSSIM simulator.

ROS workspaces: installing the simulator When working with ROS we will always work in workspaces. Workspaces are folders that contain packages, environment variables and setup files that link the scripts together. To create a workspace is very easy thing to do we need to: 1. Create a folder, the name doesn’t matter. That will be the root of our workspace.

2. Inside, create a folder called src. It will contain packages with our scripts.

3. Call catkin_make on the command line from the root.

It will create the build and devel folders with their corresponding files. Alternatively, if you are using catkin tools (like the FSSIM simulator does), you can substitute Step 3 by calling catkin init instead.

When working with C++, we will need to call catkin_make (or catkin build) every time we make a change in our scripts because it will compile them. Making a change in C++ without compiling will not change the executable. It’s really important to compile using the Release option when deploying code to be used in real time, it optimizes the execution time 100 times in some practical cases we encountered. So read carefully:

catkin_make -DCMAKE_BUILD_TYPE=Release We will always compile using Release unless debug symbols are explicitly needed, i.e. when you use the Debug option in your IDE. We discovered this too late, thanks to Miguel de la Iglesia Valls. We will discuss using C++ or Python in the future, so don’t worry much about that now.

Running the simulator: sourcing and launching If we follow the instructions of the How to Run It in your Workspace section of the FSSIM GitHub Repository, we see that running the simulator consists of two steps: sourcing and launching. What are they?

Launching: when we launch in ROS, that is, inputting roslaunch [package] [launch_file].launch on the command line; what we are actually doing is executing a series of scripts described in the launch file. So basically, launching means executing a pipeline. We can input arguments also there on the launch file. All changes done in a launch file are not needed to compile to execute again. Putting variable values in a launch file will prove useful when working with C++ too.

10

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Sourcing: when we source in ROS, that is, inputting source devel/setup.bash on the command line; what we are actually doing is telling ROS where all our scripts, launch files and packages are. So if we try to launch a script without sourcing the environment first we will get an error message such as: RLException: [XYZ.launch] is neither a launch file in package [ZYX] nor is [ZYX] a launch file name

So launching and sourcing go hand in hand. We will always need to source first and then launch. So, it makes sense to launch the FSSIM simulator by first running source devel/setup.bash and then roslaunch fssim auto_fssim.launch. It is important to source from the root of the workspace (where the devel folder is) or calling source devel/setup.bash will not find the script setup.bash.

ROS messages: moving the car with the command line How ROS works basically is by having senders and receivers - namelly, publishers and subscribers. In this concrete example, I want to send steering and accelerator commands to the car’s receiver to make it move.

Let’s see how we can do that in the easiest way. In the GitHub repository, it says: When Sending RES GO will show up in the terminal, you can start controlling the vehicle with /fssim/cmd topic.

A topic is the like URL of a receiver. So we must send our commands to the “/fssim/cmd” topic. We can input rostopic info /fssim/cmd in the command line to see what type of message it receives: Type: fssim_common/Cmd As we see, it’s a fssim_common/Cmd type of message. Let’s see the message structure by calling rosmsg info fssim_common/Cmd: float64 dc float64 delta It seems like it takes two float values. The dc value corresponds to the accelerator and the delta value corresponds to the steering. With all this information in mind we can create a simple publisher that sends the same message 10 times every second, for example: rostopic pub -r 10 /fssim/cmd fssim_common/Cmd "{dc: 0.1, delta: 0.1}" If we input this in a new terminal command line while the simulator is running, we will see the car start running in circles. Try new values for dc and delta! Negative values for delta too.

11

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Writing our own publisher: moving the car with a script It is very fun to move the car with the command line, but not only it accepts just two single values each time, we cannot do any kind of algorithm to adapt the car to go through the cones. We need to do the same we have done here but from a script.

The code below is adapted from the official tutorial website: http://wiki.ros.org/ROS/Tutorials/ WritingPublisherSubscriber%28python%29. It’s highly recommended to always search up their tutorials and learn from there too.

To understand the code line by line we suggest going to the official website linked previously.

You can download the code here: https://ideone.com/vVYOwP. Be aware that Python is really sensitive to wrong indentation, so make sure it is correctly written or it will throw a Syntax Error.

We will put this code on a Python script (.py) on the src/fssim/fssim/scripts/ folder. Let’s name it send_cms.py. As the tutorial also suggests, we will edit our CMakeLists.txt on the src/fssim/fssim/ folder input this at the end of the file: catkin_install_python(PROGRAMS scripts/send_cms.py DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}) Now, instead of calling rostopic pub we will call catkin build (once, since we changed CMakeLists.txt) and then we can call:

rosrun fssim send_cms.py to run our script alongside the simulator to make the car move. Did it work? Now, kill the send_cms.py script and change msg.dc, msg.delta values to other values you can think of.

Run the script again, does the behaviour of the car change?

Now that we are running a script we can input more complicated values. Let’s do it: t = time.time() msg.dc = max(0, 0.2*sin(t)) msg.delta = 0.1 + 0.1*cos(4*t) Try to change the msg.dc, msg.delta definition lines by these ones. Run the script again, do you see more complicated movements? That’s the power of working with scripts. Can you think of other combinations? Adding if conditions, for loops, saving values in variables and reading them on the next iteration… We can do anything!

So now we know how to create a publisher script in ROS. For now, that’s what we will need.

In the following weeks we will learn how to subscribe ourselves to the cones’ positions publisher and develop a simple control algorithm to go through them.

12

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Object detection with the YOLOv4 Neural Network We will learn what the YOLO Neural Network is and how it can be used for detecting objects (such as cones) given an image. Neural networks work given a huge dataset of labeled data, so since there are no famous public datasets (there’s a really famous private one), we will learn to detect something pretty similar: traffic signals.

We will follow a Google Colab notebook adapted from the one in the post linked below.

Post: https://towardsdatascience.com/yolov4-in-google-colab-train-your-custom-datasettraffic-signs-with-ease-3243ca91c81d Notebook: https://colab.research.google.com/drive/1849hDFjgTzGvJR6SIrDGrI-W_x6nJQyN Open the notebook and follow the steps by clicking on the play button of each code cell - in order! Note that on the Modify yolov4 architecture section, you will have to edit a script.

Here’s how you can do it:

A three step guide on how to edit the YOLOv4 architecture. Double click where the red arrows point to.

The training phase can take a long time (about 8 hours), so don’t worry if it gets cancelled because of a connection refusal (most times that happens) because it generates backup weight files every 100 iterations. If that happens, uncomment this line: !./darknet detector train data/yolov4.data cfg/yolov4_custom_train.cfg backup/yolov4_custom_train_best.weights -dont_show -map --continue-train And comment the following one. Play the following code cells to try a preliminary result of an under-trained neural network. It should deliver good results after 1000 iterations. Getting a result (whether it's good or bad) is the objective here, so don’t worry much about it.

All of this that we have done here we will now have to do it locally on our computer instead of a Google Cloud server. So the next step will be to install the Darknet and this same dataset on our own computer and do the training locally.

13

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Installing the YOLOv4 NeuralNet on our own computer This part can be done on a single computer of the team. And it should have a GPU. This is really important, GPU is at least 10 times faster than a CPU. Also if you have a Kepler or newer, it’s another 10-fold advantage. So if you can have a good GPU that’s positive.

With a good GPU (NVIDIA RTX 3060), this training can take up to 2 hours. On an older one (GTX 1050 Ti), it took 18 hours. With CPU we didn’t even want to know. Having a GPU is mandatory if we want real time performance.

Installing NVIDIA drivers, CUDA and CUDNN So the first thing we will do is to install CUDA and CUDNN (if we have a newer GPU). They are toolkits for parallel computing and deep learning developed by NVIDIA that will help us speed up our training time. They are pretty big, so let’s hope you allocated ~75GB to the root space.

This post makes it very clear: https://medium.com/geekculture/installing-cudnn-and-cudatoolkit-on-ubuntu-20-04-for-machine-learning-tasks-f41985fcf9b2. However, we will first install the NVIDIA driver: https://linuxconfig.org/how-to-install-the-nvidia-drivers-on-ubuntu -20-04-focal-fossa-linux. Following these two tutorials might require rebooting your computer a few times so make sure you are watching them from your phone.

Installing and trying the Darknet platform After installing the NVIDIA packages we will go ahead and install the YOLOv4 C++ platform.

We will download the code from https://github.com/AlexeyAB/darknet. It’s important to have the this link because if we search for “Darknet GitHub” in Google, it will show up first the original author’s repository https://github.com/pjreddie/darknet that is used for YOLOv3 and older versions.

We will go ahead and download the code by calling

git clone https://github.com/AlexeyAB/darknet.git in our command line from any folder we want to have the code there. It will create a folder with the name darknet that will contain all the code. As the GitHub repository tutorial says in https://github.com/AlexeyAB/darknet#how-to-compile-on-linux-using-make we will compile the code by calling

make on the darknet folder. But first we will change the Makefile file inside the folder by putting ★ GPU=1 ★ CUDNN=1 ★ CUDNN_HALF=1

if we installed CUDA if we installed CUDNN if we have a newer GPU

and now we can call make. If we called make before, we can call it again.

14

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

After compiling, we can follow the tutorial in the GitHub repository for training with custom objects: https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects. In our case, we will do that for the traffic signals dataset. We will follow the tutorial on the repository but instead of using obj as the name of our objects, we will use ts for our traffic signals.

We will also create two files ●

ts.names speed limit yield mandatory other

●

ts.data classes = 4 train = data/train.txt valid = data/test.txt names = data/ts.names backup = backup

and download these two files ● ●

train.txt test.txt

https://pastebin.com/S5ettTfn https://pastebin.com/qU3PysYb

and put them in build/darknet/x64/data/ as the tutorial suggests. We will also need to download the dataset in the same way as we did in the Google Colab: wget --no-check-certificate "https://onedrive.live.com/download?cid=A86CBC7F31A1C06B&resid=A86CBC7 F31A1C06B%21121&authkey=AMUUk0Np4tqH3n4" -O ts.zip unzip ts.zip rm -f ts.zip

If we put these three commands in the terminal in the build/darknet/x64/data/ folder it will create the ts folder with our dataset. Just like steps 4 and 5 of the tutorial demands.

Following the tutorial we should be able to train our neural network locally. If you experience weird behaviour while training and have CUDNN_HALF=1, it’s probably because your GPU does not support it. Change it to 0 and call make again. The same goes for CUDNN. Disabling them will cause training to last longer.

To test an image: ./darknet detect cfg/yolo-ts.cfg backup/yolo-ts_best.weights YYY.jpg 15

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Building our own cone dataset We can follow this same tutorial to detect our own cone data. If your team has already cones themselves, we can use them as data. If they are official FSG cones, you can actually contribute to the FSOCO official dataset and receive cone data from teams all around the globe: https://www.fsoco-dataset.com/how_to_contribute/. If they are not official, you can do that later when you eventually buy them and build a smaller dataset.

When building our own dataset we will need lots of variance. That means different light conditions, camera quality, point of view, background noise and settings. We will take pictures of cones in different contexts with multiple mobile phones/digital cameras.

Variance in the dataset: light conditions, backgrounds, noise...

In the example above, our team had blue and orange cones that we modified with white tape to emulate the resemblance of the official ones. The size of the dataset was of about ~1000 images with an average ~20 cones in each image.

More important than the size is (again) the variance of the dataset. So make sure you are scaling up the size while keeping different scenarios, conditions and qualities: don’t take 200 pictures of the same scenario, less than 10 in different points of view and cameras should go a long way.

To label the images (Step 5 of the tutorial), we can either use the official tool given here: https://github.com/AlexeyAB/Yolo_mark or others like https://github.com/tzutalin/labelImg. It’s important to follow best practices: https://blog.roboflow.com/tips-for-how-to-label-images/ for labeling such as labeling occluded objects, making tight label boxes and striving for quality over labeling faster. Read carefully the full best practices post linked above.

16

How to create your own Driverless Section

By Andreu Huguet (Driverless UPC - BCN eMotorsport)

Next week: ❖ Basic Control Algorithms: Pure Pursuit ❖ Darknet in ROS and Cone Distance Estimation

17