Maximizing Effectiveness of Drones in Search and Rescue Operations

Drones are increasingly becoming an essential tool for search and rescue operations. But how do we know if we are maximising their effectiveness and efficiency?

The United Nations World Food Programme has taken the lead on a global experiment to investigate.

By India Birrell, Toby Meredith, Patrick McKay and Ian Greatbatch

Introduction

Reports coming in from aircraft overflying flooded areas following cyclone Idai were ominous. Every time a pilot landed the size of the flooded area would increase by hundreds of square kilometers. Particularly chilling would be the reports of numerous people seen trapped on islands or in tree tops during an outbound flight, with the islands, trees and the people vanishing by their return. The World Meteorological Organization termed cyclone Idai "one of the worst weather-related disasters in the southern hemisphere".

In 2019, Mozambique was hit by two tropical cyclones, Idai and Kenneth, the first time that the nation had experienced two devastating natural disasters in a single season. UNICEF reported that the cumulative impact of cyclones Idai and Kenneth had caused unprecedented damage and destruction; it was estimated that 2.5 million people needed humanitarian assistance across 50 districts. Given the large-scale damage and destruction evidenced across such a large area, speed was critical to target and reach affected populations. The World Food Programme, in collaboration with government-based stakeholders, served as first responders in the first 48 hours of Cyclones Idai and Kenneth.

By using drone technology to map the disaster zone, WFP were able to effectively coordinate and distribute resources to perform emergency search and rescue operations within the immediate 48 hours. The success of WFP's mission in Mozambique highlighted the role and value of technology based solutions to improve Search and Rescue (SAR) operations. Additionally, the case of Mozambique highlighted the need for humanitarianbased organisations to take on a more productive role within emergency management activities so as to enhance the resilience of the nation state to face emergent and future disasters.

The operations in Mozambique was the first time the World Food Programme (WFP) had used drone technology in an emergency disaster setting, and was part of a broader UAS strategy by WFP within the region that was initiated in 2016, which served to converge three foundational pillars of WFP operations: specifically, imagery, delivery and connectivity.

This was evidenced in the fulfilment of coordination, search and rescue, aerial assessment, mapping and communications footage activities, which was shared with INGD, the Mozambique National Institute of Disaster Risk Management and Reduction(https://www.ingd.gov.mz), the Portuguese National Republican Guard and other humanitarian partners to deploy resources to the areas hardest hit by the cyclones. The cumulative benefits of the use and effectiveness of drones in the post-disaster recovery response to Cyclones Idai and Kenneth highlighted the potential for drones to bolster every stage of the emergency management process.

Cyclone Idai didn't follow the regular pattern typical of tropical cyclones which usually peter out when they reach land. Insead, it dropped a huge amount of rainfall, causing a wall of water to flow from Malawi into Mozambique, devastating towns and leading to a flood so large it was described as an inland ocean. This not only damaged over 200 000 houses, but also left many people stranded in trees and on rooftops waiting for rescue. Due to the scale, drone pilots were also tasked to help in the search and rescue (SAR), alongside the aircraft and boats. With wide area SAR using drones being such a new field, the efforts were very ad-hoc, with no guidelines or best practices to follow. It is this knowledge gap which lead WFP to initiate a study on the use of drones for wide area SAR.

In 2022, WFP, with funding from the European Civil Protection and Humanitarian Aid Operations (ECHO) department initiated a research project to build on the lessons learnt during the operations. This research project constituted a multi agency global SAR experiment, incorporating drones, ground forces, machine learning and artificial intelligence (AI), in multiple countries and with over half a dozen organisations participating.

The main aim of this project is to quantify the effectiveness and success rates of drones for wide area search and rescue, comparing human interpreters and machine learning algorithms to identify humans.

Obviously, many emergency response agencies are already using drones for SAR operations, and it isn't hard to find reports of drones providing essential support when searching for missing persons or providing aerial situational overview data. However, there currently exists no standardised and evidence-based guidelines on best practices when using drones for SAR, in some way due to the lack of coordinated operational experimentation.

The Experiment

The experiment commenced in 2020, with two phases designed to provide peer reviewed scientific evidence on the effectiveness and utility of drones within the development and emergency management sectors. In the first phase, human targets were randomly situated across large areas. The targets were classified as either highly visible, neutral visibility, and low visibility. This visibility type consisted of either high-visibility clothing, neutral clothing (for which the targets were asked to bring their everyday clothes), or low-visibility clothing, using camouflage uniforms of military disruptive pattern material (DPM).

The targets themselves were all volunteers from the local civil defence unit, with the exception of two manikins constructed from overalls and empty plastic water bottles, for the water based targets. The rationale for this was that snakes, crocodiles and hippos presented a risk to humans.

The drones were programmed for grid flight patterns over the areas with photographs being recorded to cover the full area at an altitude of 400 feet, and again at 300 feet. This exercise remained in line with the best practices in Search and Rescue (SAR) activities as defined by the Institute of Search and Technical Rescue, which required the experiment to be repeatable, blinded where possible and designed to isolate specific elements.

This is in comparison to a typical large scale exercise, where operators act independently without following a strict routine of actions.

In total, twelve targets were created using a random point algorithm in ESRI GIS software. These targets were distributed throughout the six search areas, known as AOIs (areas of Interest). The AOIs themselves were selected based on ground cover types selected to provide a range of backgrounds against which targets could be identified. The targets were further randomly assigned the three noted visibility types. All areas were overflown using mapping flightpath software adapted to reduce the overlap between images to 0%, and at an altitude of 90m and 122m. To compare the relative visibility and prominence of targets using RGB and Thermal imagery, additional missions were flown over the same set of targets at a 60° angle and vertically.

Images were viewed after the flight (rather than during using FPV or an observer) based on the findings of Dourtchev et al 2017 (https://journalofsar.com/ wp-content/uploads/2018/01/v2i1emergency-urban-search.pdf). Additionally, this technique mirrored the real search practices during the 2019 cyclones.

The images collected from each drone over Phase 1 were imported into a file management system created by the University of Portsmouth and then posted in a simple online software viewer developed inhouse by the WFP team. The viewer presented images to an interpreter, who then clicked a button to indicate if they thought there were any people in that image. The software recorded whether they got it right, and how long they took before giving an answer.

Once this had been done with over 100 people viewing the sets of images, the data was passed to multiple machine learning partners, who performed the same task - and critically recorded the same parameters:-

• Number of images correctly identified (successes)

• Number of images incorrectly identified as having people in them (false positives)

• The time taken to process each image.

Preliminary Findings

A novel formula for effectiveness was developed for the experiment, which sought to consider the time taken to identify targets and use this as a penalty (or reward if done quickly); and so effect, treating a slow analysis like a false positive. This formula was expressed as follows:

One outcome of using a 'time-taken' analysis is that there is now a clear difference in the effectiveness scores of the two participants who detected all the targets, with only one of them now highlighted as being most effective in targets identified and time taken. The principle here is that effectiveness drops with excessive time taken; finding all the targets but taking too long to do so is of less operational use than finding most of the targets in a short time.

In broad terms, the preliminary findings indicate a weak but positive correlation between time taken and success (defined as number of correct images identified as a percentage of the total images collected), and time taken and classic effectiveness (number of correct images identified plus false positives as a percentage) and time taken and classic effectiveness (number of correct images identified plus false positives as a percentage). This means that the longer a participant takes to review all the images presented to them, the higher the success rate is, and consequently their classic effectiveness score. However, in the case of a humanitarian or natural disaster, time remains of the essence.

Long term implications of the research

The preliminary findings produced by WFP and the University of Portsmouth indicate a substantive role for emerging and digital technologies to augment human capabilities to respond effectively in the immediate 48 hours of a disaster. The long-term value of the search methodology, and supporting formula is prefaced on its comparative assessment of the performance of future AI SAR applications, which will form a large component of the next phases of the research project. Secondly, the methodology presented may be used as a stand-alone method of determining detection effectiveness across a range of SAR research activities (for example, search dogs, detection of life signs in rubble piles or in maritime or riverine SAR). The implications of this research project elucidate a formative role for technology to augment existing human-based emergency management operations, and thus, improve the resilience and recovery efforts of local communities to bounce back after a destructive climatic event.

The organisation will release the findings of the trial later this year, which are expected to catalyse the widespread diffusion and normalisation of digital technologies as the predominant method to strengthen a nation's emergency management framework. Furthermore, if the most effective and efficient flight parameters can be identified it could have wide ranging benefits to future SAR operations and, potentially, inform the development of global guidelines, and most INSARAG approved SOPs.

India Birrell is a researcher on the use of drones for humanitarian and emergency management practices. Previously, India was the Commercial Programs Manager for the Australian drone logistics organization, Swoop Aero, where she led the integration and expansion of emerging technology into 8 new markets, working with the largest players in the global health and development sectors, including USAID, UKAID, World Food Programme, and VillageReach. She has a Master of International Relations from the University of Sydney, where she focused on the use of new and emerging technologies to design and enhance the delivery of humanitarianbased programs, including in the Middle East and Sub-Saharan Africa.

Dr Ian Greatbatch FRGS, FInSTR, FICPEM Watch Commander, Organisational Intelligence.

West Midlands Fire Service (WMFS)

Ian works on the "operations" side of Organisational Intelligence at WMFS and was previously a firefighter in the London Fire Brigade. He has also been an intelligence officer for the RNLI and was formerly an Associate Professor in a Geography Department of a UK university. He is part of the editorial team of the Journal of Search and Rescue and holds an emeritus academic position at the University of Portsmouth, UK.

Toby Meredith is a researcher on the use of drones for disaster management, with a particular focus on Search & Rescue. He is the lead drone pilot and safety officer at the University of Portsmouth and is leading the development of the drone programme for the NGO, Serve On. He has had multiple international deployments including to Dominica in the aftermath of Hurricane Maria. During the response to the earthquake in Turkey in 2023 Toby was part of the UK ops team for Serve On. He managed the drone operations and was responsible for generating essential maps of affected regions. Toby has been an integral part of the WFP experiment, designing and implementing the data management strategy, project manging the UK section of the project, presenting results at various conferences and continuing working with all stakeholders on the data analysis. In October 2023 he will be starting a PhD investigating the use of drones for SAR in UK waters.

Patrick McKay is the UAS Data Operations manager for the United Nations World Food Programme (WFP), as well as the Regional UAS Co-ordinator for the Southern African region, where he manages WFPs UAS competence centre (drone hub). Patrick joined WFP in 2004 as a mobile data collection specialist before moving into IT operations management followed by UAS operations. In 2019 he was the co-ordinator for the first official activation of drones for WFP during cyclones Idai and Kenneth.