__MAIN_TEXT__
feature-image

Page 1

VOLUME 38 NO. 1 | JANUARY-FEBRUARY 2020

UNMANNED

PACKAGE DELIVERY

ON THE COVER

SIDEWALK REVOLUTION

28

YOUR DRONE IS HERE

32

‘CUSP OF A NEW ERA’

38


The Complete Hydrogen Solution for Commercial Drones Delivering 3x the range of batteries

90 Minute Flight Time Quiet, Reliable Operation Zero Emissions Fast Refueling

We deliver fuel cell power for a sustainable planet Contact an Unmanned Systems Specialist at www.fc-air.com | (604) 454-9000


ONE INDUSTRY. ONE VOICE. Every day, Congress has the opportunity to either boost or hinder the advancement and deployment of unmanned and autonomous technologies. And every day, AUVSI works to ensure Congress is educated about the benefits our industry provides to communities and economies around the country. Join us on March 25th for the Annual AUVSI Hill Day and lend your voice to that effort. Share your story and show your lawmakers how you are bringing value to your community. Together, we will shape the future for all things unmanned.

Register now at auvsi.org/hillday

Wednesday, March 25, 2020 | Rayburn House Office Building, Washington, DC


STAFF CENTRAL

UNMANNED DELIVERY CUTTING TIME, COSTS, BUT REGULATORY HURDLES REMAIN As we enter 2020, technology is surging ahead to capture the societal, safety and economic benefits of unmanned systems. Recent advancements in the delivery of commercial goods, including the Federal Aviation Administration (FAA) granting certification to conduct air carrier operations for compensation or hire, indicate that widespread UAS deliveries are just on the horizon, as you will read in this edition of Unmanned Systems. The U.S. courier and local delivery market generates approximately $100 billion in revenue annually. The delivery of goods by unmanned aircraft systems (UAS) could help businesses reach even more customers, at lower cost, and to currently underserved markets. In addition, UAS services can help expand local economies and create jobs, while providing a sustainable, cost-efficient mode of transportation. UAS can also better reach people in remote areas with limited access to current infrastructure and provide vital resources to communities impacted and isolated by natural disasters. In other countries, we already have seen delivery times and costs reduced dramatically with drones. For example, a company in Iceland has cut delivery costs in half and shortened local delivery times by 20 minutes by using drones to transport goods over, instead of around, a large bay. And now, we are beginning to see the benefits of UAS delivery here in the United States. UPS Flight Forward has partnered with WakeMed Hospital in Raleigh, North Carolina, to deliver medical specimens across its campus as part

2

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

Brian Wynne

President And CEO, AUVSI

As has often been the case, innovative uses of drones are outpacing regulatory efforts.

of the FAA’s UAS Integration Pilot Program. Alphabet’s Wing, meanwhile, has launched a UAS delivery service at the IPP in Christiansburg, Virginia, with FedEx to deliver goods from Walgreens and Sugar Magnolia, a popular local confectionary store, directly to consumers. Amazon Prime Air, which has developed its expertise and tested its capabilities for many years, has filed for its Part 135 air carrier and operator certification from the FAA to begin delivering goods to Amazon Prime customers. As has often been the case, innovative uses of drones are outpacing regulatory efforts. For example, the current FAA rules do not allow for widespread beyond-line-ofsight operations or flights over people, and a rule on standards for remote identification is still being developed. All these regulations are necessary to permit UAS delivery in areas throughout the nation. If government policies can be implemented quickly to broadly enable expanding these operations, however, the potential benefits of UAS delivery are limited only by our imagination. Deliveries by ground robots are moving ahead as well, as companies from Amazon to FedEx to Starship are demonstrating fast, ground-based deliveries in neighborhoods and on college campuses. You’ll read about that in this issue as well. I look forward to seeing what the new year has in store and what the unmanned systems community stands to accomplish together.


Join 8,500+ users, technologists and policymakers focused on advancements in industries including package delivery, energy, transportation, construction, defense and public safety to collaborate on ideas, share lessons learned and build new partnerships.

REGISTER NOW

|

XPONENTIAL.ORG

Boston Convention and Exhibition Center | Boston May 4 – 7 | Educational Program May 5 – 7 | Exhibits JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 3


CONTENT 28

32

SIDEWALK REVOLUTION

Robots trundle into the future carrying packages and food

14

TESTING, TESTING

38

‘CUSP OF A NEW ERA’

NUAIR caps off busy 2019 with BVLOS corridor approval

YOUR DRONE IS HERE

Package delivery by drone becomes a reality

Autonomous ships set to revolutionize maritime industry

DEPARTMENTS

8 10

Industry Trends • NY governor announces drone corridor • Marine Corps seeks CRAB robots

Essential Components • FLIR tapped for CRS-H program • Whill’s personal EVs being tested

12

Company Highlight

16

Technology Gap

Optimus Ride looks operates fleet of self-driving buses

USPS seeks industry input on mail delivery by drone

22

Membership Profile

24

Viewfinder

Vivian Sun, TuSimple

Oneida Lake, New York


18

42

UNMANNED UNPLUGGED Christopher Zappa, Columbia University

Chapter News Updates from DC Capital, Saguaro and Atlanta

45

Social Update Trending news on AUVSI’s social media feeds

26

CONNECTING THE DOTS

44

FAR OUT

Multispectral imaging moves from military to the masses

MIT helps automated vehicles see around corners


2700 S. Quincy St. | Suite 400 | Arlington, VA 22206 USA P: +1 703 845 9671

F: +1 703 845 9679

Email: info@auvsi.org www.auvsi.org

BOARD OF DIRECTORS

AUVSI STAFF

Officers

Executive Team

Mark Gordon Chairman of the Board Stratom Inc. Suzy Young Executive Vice Chairman University of Alabama, Huntsville William Irby Treasurer L3 Technologies Dallas Brooks Immediate Past Chairman Miss. State/ASSURE UAS Center of Excellence

Directors 2017 – 2020 David Agnew, Dataspeed Inc. Nevin Carr, Leidos Brian Chappel, Northrop Grumman Corp. Dyan Gibbens, Trumbull Unmanned Brendan Schulman, DJI Robert Sturgell, Collins Aerospace

2018 – 2021 Ben Gielow, Amazon Marke Gibson Robert Hess, Unmanned Perspective LLC Houston Mills, UPS Susan Roberts, Panasonic Aviation Corp.

2019-2022 Sean Bielat John Coffey, Cherokee Nation Technologies Suzanne Murtha, AECOM Steven Nordlund, The Boeing Company Thomas Reynolds, Hydroid Kongsberg Jim Thomsen, Seaborne Defense

Unmanned Systems is published eight times a year as the official publication of the Association for Unmanned Vehicle Systems International 2019 by AUVSI, 2700 South Quincy Street, Suite 400, Arlington, VA 22206 USA. Contents of the articles are the sole opinions of the authors and do not necessarily express the policies or opinion of the publisher, editor, AUVSI, or any entity of the U.S. government. Materials may not be reproduced without written permission. Authors are responsible for assuring that the articles are properly released for classification and proprietary information. All advertising will be subject to publisher’s approval and advertisers will agree to indemnify and relieve publisher of loss or claims resulting from advertising contents. Annual subscription requests may be addressed to AUVSI. Unmanned Systems is provided with AUVSI membership.

Brian Wynne, President and CEO, bwynne@auvsi.org Chris Puig, Special Assistant to the President & CEO, Board Liaison, cpuig@auvsi.org Heather Lee Landers, Executive Vice President and Chief Strategy Officer, hlee@auvsi.org

Engagement Michael Wilbur, Director of Engagement, mwilbur@auvsi.org Melissa Bowhay, Associate Director of Membership, mbowhay@auvsi.org Amanda Bernhardt, Senior Manager of Chapter Engagement, abernhardt@auvsi.org Daniel Benavides, Creative Strategist, dbenavides@auvsi.org Emma Ferguson, Member Services Associate, eferguson@auvsi.org Lucy Haase, Data Integrity Specialist, lhaase@auvsi.org Sabine Hannoush, Graphic Designer, shannoush@auvsi.org

Advocacy and Government Relations Tom McMahon, Senior Vice President of Advocacy & Government Relations, tmcmahon@auvsi.org Drew Colliate, Director of Advocacy & Government Relations, dcolliate@auvsi.org Michael Smitsky, Advocacy & Government Relations, msmitsky@auvsi.org David Klein, Research Analyst, dklein@auvsi.org

Business Development Mike Greeson, Director of Sales & Business Development and Strategy, mgreeson@auvsi.org Wes Morrison, Senior Business Strategist, wmorrison@auvsi.org Alex Mann, Business Operations Manager, amann@auvsi.org

Operations Bob Thomson, Senior Vice President of Operations, bthomson@auvsi.org Karen Blonder, Director of Information Technology, kblonder@auvsi.org Kyle Culpepper, Senior IT Analyst, kculpepper@auvsi.org Mo Ahmed, IT Support Specialist, mahmed@auvsi.org Maria Ross, Accountant Manager mross@auvsi.org Librada “Rosie” Brown, Staff Accountant, rbrown@auvsi.org Anne Collins, Office Manager/Human Resources Administrator, @acollins@auvsi.org

Meetings and Conventions Staci Butler, Vice President of Meetings and Conventions, sbutler@auvsi.org Karissa Bingham, Sr. Meetings Manager, kbingham@auvsi.org Debbie DesRoches, Registration and Housing Manager, ddesroches@auvsi.org Nicole Mattar Meetings Specialist, nmattar@auvsi.org

Industry Engagement Keely Griffith, Director of Industry Education, kgriffith@auvsi.org Shannon Whitney, Education Manager, swhitney@auvsi.org

Publications and Content Brett Davis, Vice President of Publications & Content and Editor, bdavis@auvsi.org Brian Sprowl, Staff Writer, bsprowl@auvsi.org

Safety and Certification Jenny Rancourt, Certification Manager, jrancourt@auvsi.org

RoboNation Staff Contributing Authors Valerie C. Coffey, BS physics, MS Astronomy, is an American freelance science and technology writer and editor based in Indio, California. Jessica Reyes Sondgeroth is a freelance journalist and editor, with a background in energy and commodity markets, working in Washington, D.C. Rich Tuttle is a longtime aerospace and defense journalist and contributor to Unmanned Systems.

6

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

Daryl Davidson, Executive Director, davidson@robonation.org Norma Floriza, Operations Director, floriza@robonation.org Janelle Curtis, Programs & Development Director, curtis@robonation.org Hitesh Patel, Robotic Programs Director, patel@robonation.org Cheri Koch, Events Senior Manager, koch@robonation.org Julianna Smith, Outreach Specialist, smith@robonation.org Lydia Bae, Business Office Manager, bae@robonation.org David Young, Product & Training Coordinator, dyoung@robonation.org Lindsey Groark, STEM Programs Director, lgorark@robonation.org Laverne Imori, Customer Service Coordinator, limori@robonation.org Cheryl Hedeen, Community Engagement & Training Coordinator, chdeen@robonation.org


FEATURED EVENT

AUVSI HILL DAY & SCIENCE AND TECH FAIR 2020

ASSOCIATION

EVENTS

IWCE 2020 Mar 30 Las Vegas Convention Center to Apr 3 Las Vegas, Nevada

March 25, 2020 Rayburn House Office Building Banquet Rooms 2043 and 2044 Washington, DC

Want to see more events?

Visit our website auvsi.org/events or scan this code to see the complete list of upcoming events.

AUVSI XPONENTIAL 2020 May 4-7

Boston Convention & Exhibition Center Boston, Massachusetts

Automated Vehicles Symposium 2020 Jul 27 - 30

Hilton San Diego Bayfront San Diego, California

Index of Advertisers Ballard .......................Inside front cover IWCE .....................................................31 uAvionix ...............................................41


INDUSTRY TRENDS

Marine Corps seeks CRAB robots for breaching surf zones The U.S. Marine Corps plans to develop a prototype for a robotic system that would scour near-beach waters for mines and destroy them without endangering warfighters. The Crawling Remotely Operated Amphibious Breacher, or CRAB, is planned to be a submersible, autonomous system that would destroy mines with a tiller, flail or rake. “In theory, the CRAB system will breach through man-made obstacles in the surf zone,” says Capt. Anthony Molnar, Marine Corps Systems Command’s MK154 and MK155 project officer. “We will save lives and reduce costs for the Department of the Navy and the Marine Corps.” CRAB would splash into the water from a littoral utility craft and travel along the sea floor to remove explosives and other obstacles from the paths of incoming warfighters. The CRABs are intended to be expendable. Marine Corps Systems Command submitted a proposal to the Office of Naval Research for the CRAB system as a Rapid Innovation Fund topic for fiscal 2020. If approved, MCSC will develop a prototype over two years.

U.S. Marines from 1st Combat Engineer Battalion, 1st Marine Division, prepare to load an Assault Breacher Vehicle onto a Landing Craft Utility at Camp Pendleton, California. USMC would like to replace the large vehicles with expendable, mine-destroying robots called CRABs. Photo: U.S. Department of Defense.

USVI DOH, partners complete open-ocean drone crossing

SkySkopes announces spinoff company for UAS oilfield ops

In collaboration with Skyfire Consulting and Doosan Mobility Innovation Inc., the U.S. Virgin Islands’ Department of Health and the Association of Public Health Laboratories (APHL) recently completed a 43mile open-ocean drone crossing between St. Croix and St. Thomas. The UAS carried simulation vials and other health supplies in a temperature-controlled compartment during the flight. “In the past, during epidemics and following the hurricane disasters, the Department of Health had to rely on any means possible to transport critical healthrelated materials,” says Brett Ellis, director of the Territorial Public Health Laboratory within the USVI DOH. “In some cases, boats had to be used when planes were grounded. While effective, these were not the most efficient or reliable transport following a disaster.” Responders were sometimes forced to batch samples and wait for the next boat or seaplane, causing delays in potentially urgent clinical testing. Using a drone, the 43mile flight was completed in less than two hours.

Professional drone services company SkySkopes has announced the launch of SkySkopes Oil & Gas LLC, a new spinoff company for UAS-related oilfield operations. SkySkopes has launched this new division as a “single point of contact” for all UAS-based oil company needs under one umbrella. “This is an important step for SkySkopes to meet the insatiable demand for UAS data in the oil and gas industry,” says Matt Dunlevy, president and CEO of SkySkopes Inc. “This new entity is a way to bridge the gap between two of North Dakota’s most important economies: oil and aviation. I look forward to seeing the new organization flourish across the United States.” Under SkySkopes Oil & Gas, professional engineers are the subject matter experts at each end of UAS missions. To “elevate an oil company’s awareness into true decisiveness,” these engineers interpret and use the data that SkySkopes Oil and Gas collects. Headquartered in Minot, North Dakota, SkySkopes Oil and Gas also has offices in Williston, North Dakota and Fort Worth, Texas.

8

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020


U.S. DOT Secretary Elaine Chao announces the grant at a ceremony in Florida. Photo: Lake Nona

Florida area awarded grant to develop autonomous vehicle infrastructure Orange County and Lake Nona — a “smart community” within the city of Orlando and Orange County — have been awarded a new $20 million federal Better Utilizing Investments to Leverage Development (BUILD) grant that will help “reimagine mobility” in the area. The grant, the first of its kind for Central Florida, will support the planning, design and construction of a Local Alternative Mobility Network (LAMN) in Lake Nona. LAMN includes five components designed to work handin-hand to connect people to places with less dependency

on automobiles. One is infrastructure for autonomous vehicles and a bicycle transportation network. Lake Nona currently has 1.2 miles worth of autonomous network and infrastructure, but it is expected to create more than 25 miles of dedicated AV lanes. The key findings from the development and operation of the LAMN project, combined with the research and education programs sponsored by Beep — the company that operates Lake Nona’s autonomous shuttles — will help advance autonomous vehicle technologies. “As we prepare for Florida’s transportation future, these BUILD grants will provide significant investment in our infrastructure and advance multimodal initiatives here in Lake Nona and strategic areas across our state,” says Kevin Thibault, secretary of Florida’s Department of Transportation.

KDOT, Iris Automation conduct BVLOS flight with onboard DAA The Kansas Department of Transportation’s UAS Integration Pilot Program (IPP) in November announced it successfully completed the first beyond visual line of sight UAS flight using only onboard detect-and-avoid systems. This was the first time under Part 107 that the FAA has authorized a BVLOS flight without a requirement for visual observers or a ground-based radar, KDOT says. Instead, the flight employed Iris Automation’s detect-and-avoid system, Casia, which provides commercial drones with automated collision avoidance maneuvers. The nine-mile flight allowed a Kansas State University Polytechnic Campus team and Iris Automation flight team to inspect powerlines in collaboration with Evergy Inc., an energy provider in Kansas. During the first two days of flights, 150 miles were flown BVLOS. “This marks the first true BVLOS flight in the nation and is a tremendous milestone for the drone industry,” says Iris Automation CEO Alexander Harmsen. “We see this as one of the most notable accomplishments to come from the IPP program to date. We’re thrilled to set the precedent and bring our industry’s utilization of drones from dream to reality.”

KDOT flew a drone BVLOS using Iris Automation’s onboard, optical detect and avoid system, Casia. Photo: Iris Automation JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 9


ESSENTIAL COMPONENTS

FLIR tapped for CRS-H, unveils StormCaster payload family

FLIR has been tapped to build the Common Robotic System – Heavy for the U.S. Army. Photo: FLIR Systems

Nio, Mobileye partner to develop autonomous vehicles Nio, a Chinese automobile manufacturer, has announced a strategic collaboration with Mobileye, an Intel company that develops vision technology for advanced driver assistance systems (ADAS) and autonomous driving, to develop highly automated and autonomous vehicles for consumer markets in China and other major territories. Through the collaboration, Mobileye will design a self-driving system that Nio will engineer and 10

FLIR Systems has been selected by the U.S. Army to produce the Common Robotic System – Heavy, or CRS-H, based on its Kobra robot. CRS-H is designed to bolster the protection of soldiers fighting explosive devices by increasing the standoff to the weapons. The contract is worth $109 million, with deliveries expected to begin in the third quarter of fiscal 2020. “The Army is modernizing robotic and autonomous capabilities with a family of enduring systems that leverage the best of available commercial technology critical to giving soldiers overmatch in future contingencies,” says T i m o t h y G. Goddette, the Army’s program executive officer for Combat Support & Combat Service Support. “The current approach allows the Army to focus resources on fast-changing payload technology, rather than having to replace entire systems — ­ meaning soldiers can access new technology faster and can buy more of what the Army really requires.” FLIR also announced the StormCaster family of UAS payloads for the SkyRanger R70 and SkyRaider R80D drones, which it acquired with the purchase of Aeryon Labs. The new line launches with two multi-sensor products, the StormCaster-T and StormCaster-L, both of which are aimed at search and rescue missions, intelligence, surveillance and reconnaissance, and more. The StormCaster-T delivers continuous zoom and longwave infrared imaging, while the StormCaster-L provides ultra-low-light imaging, tracking, and mapping. “StormCaster-L and –T are just the beginning of this new line of advanced, high-performance payloads for our small, tactical UAS platforms,” says David Ray, president of the Government and Defense Business Unit at FLIR. “For mission needs ranging from force protection and border security to clandestine operations, FLIR StormCaster will provide heightened situational awareness, bolstered by our aircraft’s onboard, real-time artificial intelligence.”

manufacture. The companies say they will build on Mobileye’s level-4 (L4) AV kit, which provides a high degree of autonomy. According to Nio and Mobileye, the self-driving system they develop will be the “first of its kind,” as it targets consumer autonomy, and will be engineered for automotive qualification standards, quality, cost, and scale. Nio will not only mass-produce the system for Mobileye, but the company will also equip its electric vehicle lines for consumer markets, as well as Mobileye’s driverless ride-hailing services, with the technology. This variant is

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

expected to be released initially in China, with plans to eventually expand into other global markets. Nio and Mobileye have teamed to develop autonomous vehicles for the Chinese market. Photo: Nio


GA-ASI’s Gray Eagle Extended Range extends its range further with the Dynetics Small Glide Munition. Photo: GA-ASI

Gray Eagle performs flight demo with Dynetics Small Glide Munition General Atomics Aeronautical Systems’ MQ0-1C ER Gray Eagle Extended Range (GE-ER) unmanned

SenseFly announces partnerships with Trimble and Microsoft To demonstrate the capability and suitability of its fixed-wing drones and drone sensors for precision agriculture, senseFly has announced partnerships with Trimble and Microsoft. “Trimble and Microsoft are major

aircraft performed a flight demonstration in August 2019 with a Dynetics GBU-69B Small Glide Munition (SGM), a lightweight munition with increased effective range. The demonstration was a collaborative effort between GA-ASI, the U.S. Army and Dynetics. “The GE-ER operates at an altitude that provides an advantage when

using stand-off munitions,” says GAASI President David R. Alexander. “The reduced weight of the SGM allows the UAS to carry more munitions per aircraft.” The combination can effectively engage threats in a multi-domain operations environment while managing to stay out of the engagement zones of threat weapons, the company says.

players in the agriculture sector, and these relationships are a testament to senseFly’s drones operating at the high standards required by large organizations to deliver the results they need,” says Gilles Labossiere, senseFly’s CEO. “This includes being able to offer a range of cameras to meet project-specific requirements, absolute accuracy down to 3 centimeters and unparalleled flight

times that enable coverage of up to 500 hectares [1,235 acres] in a single flight.” Through its distribution agreement with Trimble, and as part of its Trimble Select list of preferred partners, senseFly’s products will be distributed through the Trimble Agriculture network to provide direct access to its products for agriculture professionals across North America and Europe.

Whill’s personal EVs being tested at North American airports Whill, developer of the Model Ci and Model A Intelligent Personal Electric Vehicles, says its autonomous, personal EVs will be tested at Dallas/Fort Worth International Airport and Winnipeg Richardson International Airport. According to the company, the trials are part of an initiative to further the current mobility-as-a-service model across the world. “When traveling, checking in, getting through security and to the gate on time is critical to avoid the hassle and frustration of missing a flight. Travelers with reduced mobility usually have to wait longer times for an employee to bring them a wheelchair and be pushed to their gate, reducing their flexibility while traveling,” says Satoshi Sugie, founder and CEO of Whill. “We are now providing an opportunity for travelers with reduced mobility to have a sense of independence as they

move about the airport and get from point A to point B as smoothly as possible. Our trials have proven to be successful in other countries and we’re excited to bring this initiative to North America for the first time.” Earlier this year, WHILL’s vehicles were trialed in Haneda Airport and Abu Dhabi Airport. They have also been trialed in Amsterdam Airport Schiphol.

Whill’s personal EVs allow users with limited mobility to move swiftly through airports. Photo: Whill

JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 11


COMPANY HIGHLIGHT

OPTIMUS RIDE LOOKS TO SOLVE FIRST- AND LAST-MILE PROBLEM WITH ITS SELF-DRIVING SHUTTLES By Brian Sprowl

An Optimus Ride self-driving shuttle. Photo credit: Optimus Ride

There is no shortage of companies developing self-driving vehicles and technologies, but few have systems that are currently operating in as many locations as Boston-based self-driving vehicle tech company Optimus Ride. Currently, Optimus Ride’s selfdriving shuttles operate in four states, including its home state of Massachusetts, as well as in Virginia, New York and California. The company’s Level 4 autonomous vehicles are designed to operate in geofenced areas such as industrial and office parks, military and academic campuses, and airports, providing first -and last-mile connectivity to public transit. “We believe that self-driving vehicles can provide that environmental benefit to reduce the number of private, internal combustion vehicles that are on the road,” said Dr. Ryan Chin, CEO and cofounder of Optimus Ride, during the groundbreaking of Halley Rise — a new $1.4 billion mixed-use district in Reston, Virginia — in October 2019. 12

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

Optimus Ride showcased its self-driving shuttles and provided attendees with a chance to experience the technology first-hand during the event. Chin noted the benefits this technology could have not only on the environment, but also in time saved, as the company has estimated the average rider of its system will save at least one week per year. “Just imagine if you had an extra week back in your life at the end of the year,” Chin said, adding that as the Halley Rise development gets larger and expands to the nearby metro, the time saved could potentially increase to two weeks.

Brooklyn bound Just a few hours north, Optimus Ride’s self-driving shuttles began operating at the Brooklyn Navy Yard in August 2019, officially marking the launch of New York state’s first selfdriving vehicle program.

Six of the company’s autonomous vehicles are being used to transport passengers between two stops, with the vehicles anticipated to transport an expected 500 passengers per day and more than 16,000 passengers per month. “Launching our self-driving vehicle system in New York at the Brooklyn Navy Yard is yet another validation that not only is Optimus Ride’s system a safe, efficient means of transportation, but also that autonomous vehicles can solve real-world problems in structured environments — today,” Chin said at the time. “In addition, our system will provide access to and experience with autonomy for thousands of people, helping to increase acceptance and confidence of this new technology, which helps move the overall industry forward. We’re thrilled to work with the Brooklyn Navy Yard Development Corporation to make history in the city and the state.”


Partnerships, challenges Companies don’t start operating this type of technology in one state — let alone four — without laying a serious foundation before deploying these systems. With this in mind, Optimus Ride says it works closely with city officials, urban planners and developers to explore the “optimal ways” in which its autonomous systems can support a particular community. “Each city and community faces its own mobility challenges; Optimus Ride takes a unique approach to self-driving vehicle implementation by working closely with city planners to introduce our vehicles in the most logical and efficient means possible,” Chin tells Unmanned Systems via email. “We work with communities to develop “autonomous ready” routes, meaning roads and networks that are optimized for self-driving vehicles, that are also intended for use by humanoperated vehicles.” The company also works closely with real estate developers to offer professional insight into autonomous vehicle readiness planning, providing guidelines for designing optimal

During the groundbreaking of Reston, Virginia’s Halley Rise, Optimus Ride provided attendees with the chance to ride in the company’s selfdriving shuttles. Photo credit: AUVSI

self-driving vehicle routes that offer a first- and last-mile option for users to their next stop or final destination. “As you design a campus, you can think of it from the perspective of an autonomous vehicle,” Chin said during the Halley Rise groundbreaking, “as opposed to trying to retrofit. And by doing that, you can design the streets differently, and design the roads differently. “Autonomous planning and readiness is very critical, especially in the planning phases of any development. And we can provide that by saving millions of dollars in excess parking and other things that we put in that you have to change later.”

‘Holistic, realistic’ With a founding team that has a combined 30-plus years of interdisciplinary experience across categories such as self-driving technologies, defense systems, urban design and smart cities, Optimus Ride says it takes a “more holistic, realistic approach” to autonomous vehicle systems by deploying its vehicles in geofenced areas and at lower speeds. The vehicles also adhere to Vision Zero speeds, meaning they operate at speeds around 25 miles per hour. “We believe adoption happens at lower speeds; by introducing the vehicles at these speed regimes, passengers are more comfortable using the technology, thus allowing us to provide more value to more users,” Chin says. By providing real users with real rides, and operating along the same route within geofenced areas, Optimus Ride says it’s able to achieve data density enabling “more meaningful insight on a self-driving vehicle’s performance and experience on those roads.” Chin says, “We can learn the nuances of the driving culture of a given area, which ultimately empowers our vehicles to perform at a more sophisticated level and ultimately making a very robust, safe, and smooth autonomous driving experience.” Safe operations aren’t just about the surroundings this technology operates in, but are also about the actual technology. To ensure safe operations, Optimus Ride announced in August 2019 it is equipping its entire fleet of self-driving vehicles with Velodyne Lidar’s lidar sensors, which provide “rich computer perception data” that enables the detection of real-time objects for save navigation and reliable operation. Using the sensors, Optimus Ride can precisely locate the position of people and objects around its vehicles, as well as calculate their speed and trajectory, allowing the vehicle’s on-board computer system to determine how to drive to its destination. “The Optimus Ride fleet showcases how Velodyne’s intelligent lidar sensors are helping companies place autonomous vehicles on the road today,” says Mike Jellen, president and CCO of Velodyne Lidar. “Optimus Ride has an ingenious approach to providing people with access to efficient and convenient self-driving mobility — effectively solving the first- and last-mile problem.”

JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 13


TESTING, TESTING

NUAIR CAPS OFF BUSY 2019 BY HELPING SECURE BVLOS APPROVAL AT NY UAS CORRIDOR By Brian Sprowl In Nov. 2019, New York Gov. Andrew M. Cuomo announced the completion of New York’s 50-mile unmanned traffic management (UTM) drone corridor, which runs from Central New York to the Mohawk Valley. This announcement came shortly after Cuomo announced that the New York UAS Test Site had received approval from the FAA to fly UAS beyond visual line of sight (BVLOS) within the first segment of the corridor — an eight-by-four-mile section of airspace between Griffiss International Airport and the New York State Department of Homeland Security and Emergency Services State Preparedness Training Center in Oriskany — thanks in large part to the efforts of Northeast UAS Airspace Integration Research (NUAIR). At presstime, NUAIR and the A drone prepares test site had flown more than to search a 2,500 test flights, all of which rubble pile during required multiple people in the NUAIR’s first New field to have a visual line of York UAS Public sight for the aircraft. Safety Fly-In. With this “true” BVLOS flight Photo: NUAIR approval, those observers will no longer be required in the field, as NUAIR and the test site have demonstrated to the FAA they can safely conduct BVLOS operations thanks to a combination of the proper safety measures and technologies. “The ability to fly with this new authority will help develop and advance many aspects of an air traffic management system for unmanned aircraft,” NUAIR CEO Michael Hertzendorf said at the time. “In order for us to fully employ, operate and unlock the true potential of unmanned systems and achieve a reality where drones are conducting routine missions such as inspecting power lines, protecting critical infrastructure, or delivering medical supplies, we need to ensure the proper safety elements are in place. This authority greatly enhances our ability to test towards that end state.” According to Hertzendorf, who was named the new president and CEO of NUAIR in October 2019 after serving as the interim CEO, the process to get this approval involved “a lot of work.” With all of that work having paid off, though, Hertzendorf believes the approval and the subsequent testing will be a major stepping stone for the industry and the future of these operations. “It gives us leverage to start testing and gathering data at a much greater level to understand the 14

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020


challenges and gain community acceptance,” Hertzendorf tells Unmanned Systems. “It’s a very necessary first building block.” The result of a $30 million state investment, the 50mile drone corridor — considered the first of its kind in the nation — will provide companies with the space to test UAS and UTM technologies in real- world settings to collect data that will help inform the industry and regulators, and push society toward regular commercial drone use. “The completion of the 50-mile drone corridor is a groundbreaking achievement that caps a key strategy laid out in our CNY Rising plan to make Central New York and the Mohawk Valley a global center for UAS testing and innovation,” New York Gov. Andrew M. Cuomo said at the time. Hertzendorf says the corridor will continue to adapt, mature and develop as more technology comes down the pipeline. Nonetheless, the corridor will help the state of New York in its quest to be a leader in the industry. One of the things NUAIR wants to do, Hertzendorf says, is help launch programs that benefit state agencies and help them solve problems. NUAIR also wants to direct those agencies to potential partners to ultimately develop the best use cases for this technology that will positively impact the state. For Hertzendorf, collaboration will be a key factor in helping push the industry forward. “Nothing done in this industry is done in isolation,” Hertzendorf says. “Everyone has to be cooperative in this industry, and NUAIR wants to continue to [facilitate that collaboration] to advance the industry.”

Busy year New York Gov. Andrew M. Cuomo announces the completion of the New York’s 50-mile unmanned traffic management (UTM) drone corridor. Photo: NUAIR

The year 2019 was a busy one for NUAIR. Among its many activities and initiatives, NUAIR hosted the second annual New York UAS Symposium in September, which brought together more than 300 participants and provided them the opportunity to talk with fellow UAS industry leaders representing all sides of the industry, make new connections, and discuss key issues that the industry is facing. Shortly after hosting the symposium, NUAIR hosted its first New York UAS Public Safety Fly-In in October, which brought together more than 100 public safety officials from across New York State to the State Preparedness Training Center (SPTC) in Oriskany, New York. During the event, participants got the chance to learn about and fly UAS in a variety of emergency scenarios that were set up throughout the SPTC. Other notable events and achievements for NUAIR during the year included its participation in the first-of-its-kind evaluation of remote ID for drones, the establishment of its partnership with the Oneida Indian Nation, and its validation of a drone parachute for package delivery company Flytrex. As Hertzendorf says, 2019 “was a big year for us.” JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 15


TECHNOLOGY GAP

US POSTAL SERVICE SEEKS INDUSTRY INPUT ON

MAIL DELIVERY VIA DRONE By Brett Davis

Eventually, neither snow nor rain nor heat nor gloom of night will stay U.S. Postal Service drones from the swift completion of their appointed rounds. The USPS recently issued a request for information from industry about unmanned aircraft, specifically those that could be used for letter or parcel delivery. “The purpose of the RFI is to seek information from industry experts on unmanned aircraft systems (UAS) for the purposes of research and development in this area,” the service says in its only statement on the effort, and a spokesman said there won’t be any further comment at this time. The service says it is “investigating the feasibility” of using drones “as an integrated part of its vehicle delivery fleet, as well as to provide image and other data collection services.” Although the USPS warns it does not plan to award a contract on the basis of the RFI, it does highlight ways it could see the systems being used. 16

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

Various uses The U.S. Postal Service is seeking industry input on mail delivery via drones. Photo: iStockphoto

Drones could aid with what the USPS calls “long driveway delivery,” where a drone launches from a vehicle, makes a delivery and returns to the vehicle. Something similar was demonstrated by UPS and Workhorse Group in early 2017, where a van served as a launch platform for a delivery drone. That test was conducted in Lithia, Florida, with Workhorse Group, which built both the truck and the drone. “This test is different than anything we’ve done with drones so far. It has implications for future deliveries, especially in rural locations where our package cars often have to travel miles to make a single delivery,” Mark Wallace, UPS senior vice president of global engineering and sustainability, said at the time. “Imagine a triangular delivery route where the stops are miles apart by road. Sending a drone from a package car to make just one of those


deliveries can reduce costly miles driven. This is a big step toward bolstering efficiency in our network and reducing our emissions at the same time.” USPS says drones could also aid in delivery to remote or rugged locations, such as small islands or over rugged terrain. In addition, USPS notes they could serve as a “ride sharing model,” where businesses use USPS drones to deliver their products, or where drone service providers use USPS drones for nondelivery tasks such as power line inspection. Based on the level of interest from industry, the Postal Service says it may identify companies for a future solicitation to develop UAS operations and demonstrate their safety and reliability. “The Postal Service’s investigation is focusing on developing solutions for remote piloted aircraft operations for delivery of mail beyond visual line of sight, as well as developing universal standard navigation capabilities for UAS, secure data protocols, and best practices for maintenance and training programs in the UAS arena,” the RFI announcement says. The service says eventually the drones may be used on specific routes that have high operational costs, where drone-related cost savings would come in handy.

All weather

Scan this to see UPS and Workhorse demonstrate drone package delivery.

In 2017, UPS and Workhorse demonstrated a topic the USPS is seeking data on, namely the use of trucks to serve as launch platforms for drone deliveries. Photo: UPS

“They will be operated in all FAA-approved weather conditions found in the United States,” the RFI announcement says, which it defines as from minus 40 degrees to plus 140 degrees Fahrenheit, with a relative humidity of 5 to 98 percent. This would include extensive testing in “real world delivery environments identified by USPS,” and “the overall testing goal is to bring a valued and safe service to the United States that is expandable and sustainable. “UAS will be operated for a minimum of eight continuous hours per day with exchangeable batteries, over hilly terrain, water, semi-improved roads, and, eventually, over city streets.” They will make an average of four flights and deliveries per day of mail and packages but could do as many as three per operational hour, it notes. “The final objective is to validate the aircraft operations via repetitive successful completions to provide the confidence necessary to be awarded a Type Certification of an aircraft as a UAS for delivery of mail,” the RFI announcement says. “This will allow a UAS program to deploy nationally.” Final submissions to the RFI were due by Nov. 4.

JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 17


UNMANNED UNPLUGGED

Christopher J. Zappa is the Lamont Research Professor at Columbia University’s Lamont-Doherty Earth Observatory, where he leads the Observatory for Air-Sea Interaction Studies, employing both drones and unmanned surface vehicles. This interview has been edited for length. Scan the bar code to watch the full interview and hear his thoughts on getting a certificate of authorization to fly large drones, his need for a sensor to measure fog, his next research journey, and other thoughts.

Q: Could you tell our viewers and readers a little bit about your work and how you came to get into that? A: I am a physical oceanographer here at Columbia University’s Lamont-Doherty Earth Observatory. My background is actually as an undergrad, I was a mechanical engineer, so have always been proficient at building things and wanting to build gadgets that observe and sense the environment. I went off to graduate school after undergraduate. I was interested in the ocean, obviously went into oceanography ... but the kind of oceanography that I do is atmosphere-ocean interaction. And it’s a very dynamic environment, so you need very cutting-edge sensing systems. So, I’m always thinking of new ways to probe or observe the air-sea interface where there’s breaking waves, waves, the surface is moving up and down very rapidly. Early on I did a lot of imaging, solar infrared imaging, polarimetric imaging, measuring atmospheric boundary layers, wind speed, turbulent wind, and so measuring the small-scale perturbations in the atmosphere, measuring the turbulence in the upper ocean. And then I would say about 10 years ago I started to get into UAVs. I first started working on a NASA project up in the Arctic and we basically took the infrared imaging capability that I would usually put on a ship to look at the ocean and I started to fly it from an aircraft manned aircraft. Now I’ve started and I just miniaturized it and made a payload to put on a ScanEagle. That was the first UAV we used, an Insitu ScanEagle. Since then I’ve expanded the instrument payloads that we use, we’ve developed with funding grants from the [Gordon and Betty] Moore Foundation and from Schmidt Ocean Institute. After the first experiment up in the Arctic, the Moore Foundation contacted me and said that they are interested in developing more payloads. So, I basically miniaturized every facet of what I would I do from a ship and I basically miniaturized all those sensors and put them on aircraft now, UAVs. We did that from the Arctic again, that was from land, and then the Schmidt ocean Institute said, well, look, we want to do this from a ship. 18

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

Christopher J. Zappa

Scan this to watch the complete interview with Dr. Zappa.

So, I adapted all my payloads and put them on a new kind of system, I don’t know if you’ve heard of it, from Latitude Engineering, now L3 Latitude ... [it] was bought up by Harris, but they have this vertical takeoff and landing capability, and it’s ideal for working from a ship. So, we did that, showed that we can do that from a ship, on the R/V Falkor, which is the Schmidt Ocean Institute’s ship. The [Moore] board foundation came back and said, well, we really like this idea. We want to do this from land again, but up in the Arctic. And so, we did a whole two-year project up in the Arctic using the UAVs to look at sea ice and sea ice changes in and around the Alaska coastline, and then the Schmidt Ocean Institute came back and said, oh, we wanted to do this next phase, which is where I went to them and said, we want to do long-range telemetry of the data. So now, this past five months, six months, we’ve upgraded the UAVs, so I have of these HQ-90s from Latitude. We’ve upgraded them now, so they, all my payloads fit on, adapt or are fully adaptable. I just pop the nose cone on and off with a different payload. And now they all have both dual GPS, so that they can take off and land autonomously from the ship, and they all have long-range data capabilities. So that all the payloads that I have on there, telemeter all the data back, up to 50 nautical miles away. Q: So, you’re getting your data just in real time.

Scan this to see Zappa’s crew conduct an autonomous drone launch from a moving research vessel.

A: Real time. We used to have an idea where we want to go and target a certain feature. So, say we’re looking at the ice age of the sea ice and we get a satellite image and hopefully the ice hasn’t moved around and we’d go back and map out that area in higher resolution and sometimes we’d get it, sometimes we wouldn’t, because the ice moves, right? So, this allows us to get that satellite image, have an initial target. If we go out and we map an area and then we’d say, oh, it’s really over here, we can reprogram it to go and do a new flight track where we


really want it to go. So, you’re not wasting the flight hours and you’re getting much betterquality data because you can see it coming back real time and make changes real time to the system, that will optimize the data.

The L3 Latitude VTOL UAS flies in the Arctic on a research mission to study sea ice. All photos: Lamont-Doherty Earth Observatory

Q: You mentioned working from a boat previously, so before you were using drones, how would you make all these different observations? A: That’s an excellent segue to the next thing I would want to say. What we used to do is just, I would mount them literally to the highest point on the boat, on the ship, look off to an area where I could, when it wasn’t disturbed, which made it very difficult. And plus, the ship is heaving. So, there’s lots of motion correction you have to do for the imaging, and also for the atmospheric modular measurements. So, now, two things from an aircraft — you both get away from the ship, so the superstructure of the ship isn’t disturbing what we’re trying to measure and two, the platform is much more stable than on a ship. So, we get the benefit of two things: You don’t have the superstructure disturbing our measurements and ... we’re also on a much more stable platform for our measurements as well. Both of those are a big, big advantage. Q: Have you worked with any unmanned underwater systems or unmanned surface systems? A: I do, yes. Good question. Definitely surface systems. Before I did unmanned aerial vehicles, I was doing many different types of surface vehicles. Very crude, not off the shelf like the fancy Wave Glider systems now, and the Sail Drone, and all these kinds of things, that are great. In the past, we would develop our own kind of surface [systems], either drifters or buoys, but the autonomous vehicles we would build, we’d have catamarans. And basically, the first catamaran we built was based on an old Hobie Wave. We took a Hobie Wave and we’d put some propellers on there and autonomous propulsion ... not fully autonomous, let’s say remote, so we could steer the catamaran where we want to go, and put all the instruments on that catamaran. We did a few generations of these catamarans. So, on this cruise that we have coming up, we also have catamarans, and drifters, and the UAVs. Underwater vehicles? I, myself, have collaborated with people that used gliders, mostly gliders because they probe the near surface, top 200 meters of the ocean, which is what I’m mainly interested in. Most of the remotely operated vehicles kind of go deeper and I stay right near the surface. JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 19


Q: You said you’ve been building and miniaturizing your own payloads. Are you just not able to find what you need on the commercial market or are they just too big? A: Both. Most of them. So, the cameras, most of the visible cameras I find on the commercial market. The infrared cameras, I always push ... I work with specific companies. Sofradir is one company I work with. I’m not trying to plug anybody, but I work with Sofradir, and the beauty of working with a company like that is they build the sensors. And they’ll build the engines. And I keep telling them, I want it smaller, as small as you can get me, a cooled sensor, [the smaller] the better. So, I’ve always pushed them or we’ve worked somewhat in tandem to get the sensor as small as possible for my purposes. Q: What advice would you give to researchers who maybe want to use drones and aren’t sure how to get started? Q: What is the importance of understanding the sea-air interfaces? Is it weather related, climate related, what do you get from it? A: It definitely depends on who your funders are. In general, in understanding the ocean-atmosphere interface, we are trying to understand how energy or momentum from the atmosphere in the form of wind generates waves and how much of that energy gets into the ocean. That is important for weather forecasting ... short-term weather forecasts. What I do is most closely related to weather forecasting, short term, your 10-day forecasts that you see, marine forecasts, the understanding that I provide from the data that we get, we will develop models which will then feed into the marine forecasts. I would say another level of what we do is understand exchanges of heat into the ocean and specifically, marine atmospheric gases like carbon and methane, specifically carbon. I try to model how much, how fast, carbon goes into the ocean or goes out of the ocean. That would be useful for climate models, which are much longer term, hundred-year models, multi-decade to 100-year duration models.

Zappa’s research crew in the Arctic.

20

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

A: Well, I was going to say, let me do it for you! It is a great undertaking. It really is. Working with drones, you really have to be sure that it is what you need as your tool. You know, I got into this, basically in the past 10 years I’ve been doing development. I’ve been developing for 10 years now to get to the point where I am, now I have a tool that I can use for real science. ... These are not just your octocopter kind of systems that go out for 15 minutes. You can get octocopters now that have the same size and you can put the same size payload that I have, but the drones I’m talking about, they go out for 15 hours. I send them up, they go out, they fly over the ocean, they map out hundreds of nautical miles of real estate ... I can task to go all over all over the place for over 15 hours. These are high endurance, high payload systems. They’re not an easy thing to just do. It takes a lot of both ground support for the UAV itself and a lot of payload support. ... For instance, on this next cruise we’re going on, we’ll have three pilots from the company that have been contracted, and there will be five of us on the payload side, one specific payload specialist, two people doing ground support, helping the pilots, then two people doing the real-time tasking ... so it’s not a small operation, but it depends on how much you want to get out of the system.


Learn from industry thought leaders with AUVSI’s Webinar Series without ever leaving your office. Each month experts share their insights and answer your questions about the unmanned community’s most pressing issues. Webinars are FREE for AUVSI members and $49 for non-members. Learn more about becoming an AUVSI member at auvsi.org/why-join-auvsi

Sign up for an AUVSI webinar today! auvsi.org

Connect with us:

Association for Unmanned Vehicle Systems International

JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 21


MEMBERSHIP PROFILE

A few questions for industry movers and shakers The technology I’m most excited about is: The technology that I’m most excited about is self-driving trucks, because they address real business problems, such as the growing driver shortage in the trucking industry. Self-driving trucks also solve other issues in areas that I am passionate about, such as minimizing greenhouses gases emissions and reducing human-caused errors on the road to increase safety. My favorite robot movie is: Transformers. Something about a class-8 semi transforming into a 20 feet robot just really resonates with me. The best advice I ever got is: Don’t care about what other people think or say about you, just be yourself. For fun, I like to: I like to spend time outdoors with my dog. We like to enjoy the sun and the beautiful beaches around the area. I am blessed to be living in San Diego, where the weather is always nice!

22

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

When I was a kid, I wanted to grow up to be: I wanted to grow up to be a successful business woman, as there’s a role model for me in my family. The best part of my job is: The best part of my job is facing new challenges every day. There is never a dull moment and I love every part of it! The strangest thing in my office is: A pair of Ping Pong rackets. Not sure when this was part of my office decoration, but it has been sitting on my desk for the past year.


Vivian Sun Senior Director of Business Development, TuSimple

JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 23


VIEWFINDER Reader Frank J. Dahl II sent this photo, taken by a Mavic 2 Pro at 144 feet over Oneida Lake in upstate New York. Have a great photo you’ve taken with an unmanned system? Send it to Brett Davis at bdavis@auvsi.org

24

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020


JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 25


THE DOTS SPECTRUM EVOLUTION: MULTISPECTRAL IMAGING MOVES FROM MILITARY TO THE MASSES By Valerie C. Coffey Multispectral imaging obtains information from several bands of the electromagnetic spectrum to reveal a wealth of information, enabling differentiation of objects that cameras cannot. We can credit astronomers for laying the foundation of multispectral imaging (MSI) technology, perhaps starting with the well-known prism experiments of Isaac Newton in the late 1600s, followed by the experiments of German astronomer William Herschel. In 1800 Herschel was surprised to find that a thermometer held just outside the visible red part of the spectrum was warmer than that of visible light. American astronomer Samuel Pierpont Langley invented the first bolometer in the late 1800s to measure thermal radiation in the infrared (IR) part of the spectrum more sensitively. He used a telescope along with his platinum

foil bolometer instrument to measure how much solar energy reaches the Earth at far infrared (FIR) and sub-mm wavelengths. The track of MSI history closely follows the development of better and better detectors. In the early 1900s, inventors filed numerous IR patents, including a thermopile “infrared eye,” developed to detect icebergs in 1913 shortly following the Titanic tragedy. In the 1930s, the German, American, and English governments developed the first IR photon detectors using leadsulfide (PbS) and lead selenide (PbSe) thin-films. These detectors enabled scientists to “see” beyond the visible spectrum (VIS) out to IR wavelengths of 3 um, and understand the heat signatures of our environment. These first multispectral imaging experiments established the elemental and molecular “fingerprints” of emission

and absorption produced in vibrating atomic bonds. VIS images overlaid with information from other spectral bands held potential to enable object recognition, navigation at night, and troop movement. Thus, research in IR detection was primarily the purview of military and government laboratory research through World War II. The first systems were large, bulky and very costly. In the 1950s, military technology began its transfer to civilian applications, such as astronomy and meteorology. The TIROS 2 weather satellite, launched by the U.S in 1960 with television cameras and IR radiometers, was the first multispectral imaging system in space. The development of semiconductor alloys in the II-V, IV-VI, and II-VI groups of the periodic table in the 1950s enabled custom tailoring of photo detectors with tunable bandgaps. With

A complex region of forest, stream, farmland, and water in visible light (left). The same region imaged in the red, green, and red-edge bands of the RedEdge-MX Dual Camera imaging system reveal chlorophyll efficiency, while the blue aerosol band monitors shallow water environments for algae and other plant growth (right). Image: MicaSense

26

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020


the introduction of these new types of materials for detectors, commercial interests began to energetically develop smaller, more costeffective MSI technology. The most widely used material for tunable bandgap detection today is mercury cadmium telluride (MCT or HgCdTe), a group II-VI material. Second-generation detectors came on the heels of the computer revolution, introducing mass-produced silicon charge coupled devices and complementary metal-oxide-semiconductor chip technology. In just the past few years, advances in analytics, smart sensors, and tiny photonics integrated circuits have improved processing time and performance of MSI systems. Detectors using new semiconductor detector materials like indium gallium arsenide (InGaAs) and antimonide combined with electronic readouts have enabled focal plane array designs that reduce size, weight, and power. These developments have greatly expanded MSI to numerous consumer and industrial applications, including maritime safety, oil and gas, machine vision, biomedical, and food safety. Art conservation was an early adopter of MSI in the early 1990s. High-resolution MSI sensing of reflected light on paintings across the spectrum from UV to IR is non-invasive and can reveal hidden details beneath layers of antique paint, like moisture damage, brush stroke techniques or the original artist’s pencil sketching.

Moving upward Drones equipped with multispectral imaging systems have become useful in many recently developed commercial applications, such as archaeological, mining and construction surveys. Agriculture in particular has quickly seized on the advantages of MSI via drones, enabling farm managers to detect elevation variations in a field to help with crop selection, monitor moisture levels in soil and crops to help identify drought- and flood-stressed areas, and guide irrigation and flood remediation efforts. Bird’s eye multispectral imagery can also monitor emerging crops, provide guidance on fertilizer application, and identify weeds, soil compaction, salinity, and nutrient deficiencies. According to Drew Baustian, business development manager at agricultural drone sensor company MicaSense, headquartered in Seattle, one of the biggest hurdles facing multispectral drone sensing is interpreting the images and spectral data obtained. Farm managers have to learn what specific spectral results mean and which practical actions they indicate. The spectral bands driving most drone imaging today are the same ones chosen for the Landsat and Sentinel series of satellites — red, green, and blue (RGB), red-edge, and NIR bands. These bands are key in measuring plant reflectance, so have helped researchers establish normalized difference vegetation index (NDVI) indices for farming, which distinguish characteristics of crops and vegetation. As such, the visible to NIR bands form the basis for most published agricultural research today. “The spectral bands in MSI cameras today are designed to cover the same wavelengths the Landsat 8 and Sentinel 2A satellites do, so farmers can compare drone data with

From a variety of airborne platforms, the MicaSense RedEdgeMX Dual-Camera imaging system integrates two five-band cameras, the RedEdge-MX and RedEdge-MX Blue, to obtain ground images that enable direct comparison to Landsat and Sentinel satellite data. Image: MicaSense

satellite reference data and apply established research to their own crops,” Baustian says. “What we’ve learned about plant reflectance from satellite imagery can now be leveraged by individual farm managers using drone imagery.” The future of agricultural MSI lies in working with academic institutions to hone in applying the raw data to a specific crop management practice, Baustian adds. Numerous earlystage drone software companies like Agremo (Serbia), Ceres Imaging (Oakland, California), Pix4Dfields (Berlin, Germany), and Delair (Toulouse, France) are taking the next steps to develop recommendations for specific crops, such as a tool to help determine when to apply a growth regulator to cotton so it doesn’t grow too tall to harvest. In addition to the use of drones to obtain a view of Earth from above in multiple spectral bands, numerous companies are focused on launching space-based platforms to establish commercially available multispectral imaging services. These micro- and nano-satellites provide multispectral imaging as a service from a fleet of satellites the size of small breadboxes ringing the Earth. San Francisco-based Planet Labs, one of the most prominent Earth imaging providers, has over 200 tiny satellites in its orbiting constellation as of late 2019, mapping most of the landmass and coastal regions of Earth once a day at 3 to 5 m optical resolution in the RGB, red-edge, and IR bands, with plans for many more satellites to be deployed in future launches. With this business design, customers can pay to access a database of historic survey data in numerous bands, or they can hire an imaging satellite to look at a particular field or part of the ocean at a certain time of day or year. Analysts predict a $9 billion market by 2027 will tempt many more companies to jump on the multispectral Earth-imaging as a service bandwagon. What used to be the exclusive realm of secret militarygovernment projects is now available online (in a controlled interface, with some limitations) to researchers, academic institutions, and well, just about any paying customer. The implications of selling potential secrets are profound, exciting, and most certainly complicated. JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 27


COVER STORY

SIDEWALK REVOLUTION:

DELIVERY ROBOTS TRUNDLE INTO THE FUTURE, CARRYING PACKAGES AND FOOD By Brett Davis and Brian Sprowl Near a statue of Founding Father George Mason, we wait for the future to arrive. We opened an app on our phones, ordered a couple of pizzas and waited a few minutes. The app said the delivery was on the way, and shortly thereafter a little white robot came trundling down the sidewalk at George Mason University, moving slowly but steadily on its six wheels. “Hello! Here’s your delivery!” the robot said, after we clicked the app to notify it we were nearby. We withdrew two hot, fresh pizzas and sent the robot on its way. The robot was from Starship Technologies, which has moved out in a big way into the robot food delivery market. The robots made their debut at George Mason in January 2019, the first day of the spring semester. Students weren’t told the robots were coming — they just showed up. “It was very much a pilot at that

28

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

point,” says Ryan Tuohy, the senior vice president of business development at Starship Technologies. “It was our first university deployment, so we installed the robots while the students were on break, to have it be a surprise when they returned.” Mark Kraner, executive director of campus retail operations at GMU, says, “we turned them on and really did minimal marketing, but the students found them, they found the app, downloaded the app, we started building deliveries from day one, and it just took off.” Since then, the students have embraced the robots, Kraner and Tuohy agree. Students dress them up for events like Valentine’s Day, put “thank you” notes in them, name them and pose with them on social media. The most popular photo from this year’s graduation is a student posing with a robot, Kraner says. The students are also protective

of the robots and make sure they can get through on sometimes crowded sidewalks. “They know what it’s doing, it’s bringing someone their food, or their Starbucks,” Tuohy says. “After the first couple of days of novelty, they treat it like any other pedestrian, allow it to make its way through the crowds right along with the humans.” While there is a “very small percentage” of people who are suspicious of robots, “the majority thinks they are the best things in the world,” Kraner says. “It makes their life easier.” Last summer, Starship Technologies announced it had closed $40 million in Series A funding and planned to use it to expand to 100 university campuses in the next two years, going well beyond George Mason’s leafy campus. They have already launched at Purdue and the University of Wisconsin, among others, and job postings on


the company’s website seek operators and support staff in college towns such as Austin, Texas, and Tuscaloosa, Alabama. Starship Technologies was launched in 2014 by Skype cofounders Ahti Heinla and Janus Friis. Like Skype, it was founded in Estonia but has since branched out, with its headquarters in San Francisco and offices around the world. By the time the robots began traveling around George Mason, they had already racked up 100,000 miles in 20 countries and 100 cities, Tuohy says. “This company did not rush to market. Our founders had the vision to start early, and the resources and patience to get it right before we went out with the robots.” The robots rely on a combination of machine learning, artificial intelligence and sensors to get around. They specifically don’t use lidar, as it would make the cost of delivery too expensive. As they continue to operate, their learning gets better. “The more we drive, the better we get,” Tuohy says. They’ve been so successful that George Mason has expanded robot delivery hours and is mulling further expansion, possibly adding book and other package deliveries and expanding the service to surrounding neighborhoods. Tuohy says adding package delivery and even services such as laundry would be easy to do, “but the demand for the food delivery has been so high we’ve really been focused on keeping up with this core business.”

Competition According to a recent study from Markets and Markets, the delivery robot market is expected to grow from $11.9 million in 2018 to $34 million by 2024 (by contrast, the same company says the drone delivery and logistics market will hit $11 billion by 2022 and $29 billion by 2027). The market is not large by comparison, but Starship is only one of several companies deploying or developing ground delivery robots, and more contenders are entering the field. Last summer, Amazon began testing its Scout delivery robot in Irvine, California, after an initial test run in Snohomish County, Washington, just north of Amazon’s home of Seattle. A small number of robots are now delivering packages, working Monday through Friday in daylight hours. They can deliver food or anything else from Amazon a customer might order, as long as it fits inside the robot. As was the case in Washington, Amazon users might get a robot delivery or they might get a human delivery. Even the robot deliveries are accompanied by what the company calls an “Amazon Scout Ambassador” to help answer questions or allay any fears a customer might have during an unexpected encounter with a robot, even though Scout is not particularly threatening — it looks like a sixwheeled, blue cooler and rolls along at a walking pace. Amazon created dedicated hardware and software labs in Seattle to create Scout, allowing them to quickly build and test the robots. Sean Scott, Amazon’s vice president of the Scout program, wrote in a company blog that he’d like to bring Scout “to the rest of the world, and one software engineer said it would be good if Scout could talk.

Package and food delivery via ground robots may be a relatively small market in the near term, according to industry analysts, but it’s already attracting a number of competitors using a wide variety of systems.

FedEx's SameDay delivery robot, dubbed Roxo, made its public debut in early 2019 on Jimmy Fallon's Tonight Show. Photo: FedEx

JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 29


FedEx’s Roxo In February 2019, FedEx Corp. unveiled Roxo, the FedEx SameDay Bot, an autonomous delivery robot designed to help retailers make same-day and last-mile deliveries to their customers. Roxo made its debut on the Tonight Show with Jimmy Fallon, where it brought him a pizza. “Look at this thing!” Fallon said. “This is the future!” According to FedEx, retailers will be able to accept orders from nearby customers and deliver them directly to customers’ homes or businesses the same day using the robot. FedEx notes that on average, more than 60 percent of merchants’ customers live within three miles of a store location, which shows the potential for on-demand, hyper-local delivery. With this in mind, FedEx has also announced that it is partnering with several companies, including AutoZone, Pizza Hut, and Target, to help assess retailers’ autonomous delivery needs. DEKA Development & Research Corp. helped develop the FedEx bot. The company was founded by Dean Kamen, who has invented technologies such as the iBot Personal Mobility Device and the Segway. Roxo also has six wheels, but two of them are smaller. It also rides higher than the Starship or Amazon delivery devices. With its navigation system on top, it looks less like a cooler and more like a robot. Thanks to proprietary technology, the bot can also navigate unpaved surfaces, curbs, and even steps for an “extraordinary door-to-door delivery experience,” FedEx says, a vestige of its Segway roots, as Kamen years ago built wheeled systems capable of climbing stairs. 30

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

A Starship food delivery robot makes its way across the George Mason University Campus in October, 2019. Photo: AUVSI

Scan this to watch a Starship Technologies deliver a pizza at George Mason University in Virginia.

Future jobs “The FedEx SameDay Bot represents the next chapter in our long legacy of delivering innovation and outstanding service, supported by an already existing FedEx logistics ecosystem,” said Brian Philips, president and CEO of FedEx Office. “We are excited to bring this technology to address new markets and better support our customers. The companies who have provided feedback on its potential use have been instrumental in ensuring we are looking toward the future of e-commerce.” The future of e-commerce also means creating new types of jobs on campus. Starship’s Tuohy says a small staff is required on each campus where the robots operate, and the company hires students. “We try to create jobs for students who are into autonomous navigation, computer science majors, some of the mechies. We intentionally create parttime jobs that allow these people to get under the hood of this technology.”

Other bots Markets and Markets notes there are several other large players in the space as well, including Savioke, Nuro, Boston Dynamics, Robomart, Dispatch (since acquired by Amazon), Piaggio Fast Forward, Marble, BoxBot and KiwiCampus, all in the United States, as well as JD.com in China, Panasonic System Solutions Asia Pacific in Singapore, Eliport in Spain and TeleRetail in Switzerland. Other companies are edging into the delivery market in addition to these, including food delivery company PostMates, with its four-wheeled Serve robot, and Segway with its Loomo, a version of which delivered a cold drink to Brian Krzanich, the then-CEO of Intel, when he made a keynote address at AUVSI’s Xponential event in 2017.


OADBAND—PUBLIC

AND

PRIVATE

SYSTEMS

|

MISSION

CRITICAL

TECHNOLOGIES

OPERATIONAL,

NEFITS | LAND MOBILE RADIO (LMR)—PUBLIC-SAFETY, CII AND ENTERPRISE | 4G/5G WIRELESS--LON

OLUTION (LTE)/WI-FI/WIMAX | NETWORK DENSIFICATION/SMALL CELLS | IN-BUILDING COVERAGE—DAS

CKHAUL/FRONTHAUL—FIBER,

MICROWAVE,

SATELLITE,

ETC.

|

CBRS/PRIVATE

LTE

NETWORKS

|

MI

ITICAL PUSH-TO-TALK (MCPTT) | PTT OVER CELLULAR (POC) | DEPLOYABLE NETWORKS—VEHICLES/

WS/CARRYING

CASES/BACKPACKS

|

DRONES/UNMANNED

VEHICLES

|

WIRELESS

SITES

AND

TOWERS/N

FRASTRUCTURE FORUM | CALL CENTER/DISPATCH—ON-PREMISE/CLOUD-BASED | 911/NG911 | COMPUTER

SPATCH

(CAD)

|

ANALYTICS—DATA,

LOCATION-BASED VIDEO

AND

VOICE

SERVICES |

(LBS)/MAPPING—GPS/GIS

ARTIFICIAL

INTELLIGENCE

|

(AI)

CONNECTED |

FIBER

|

COMMAND VIRTUAL

C

RE

GMENTED REALITY | SDN/NFV/SON | APPLICATION DEVELOPMENT | EDGE COMPUTING | WEARABLES

ORAGE--BIG DATA/CLOUD/FOG | SMART/SAFE CITIES | INTERNET OF THINGS (IOT)/INTERNET OF LIFE

INGS (IOLST) | CYBERSECURITY | NETWORK RESILIENCY/AVAILABILITY | HARDENING/PHYSICAL SECU

WER—BATTERIES/GENERATORS/FUEL

ONTHAUL—FIBER,

MICROWAVE,

CELL/SOLAR/WIND

SATELLITE

|

|

SYSTEM

NETWORK

MANAGEMENT/MAINTENANCE

INTEROPERABILITY

AND

|

INTEGRATION

BAC |

H

TEROGENEOUS NETWORKS (HETNETS) | IT AND COMMS INTEGRATION | SPECTRUM AND SIGNAL OPTIMIZA

SE STUDIES AND BEST PRACTICES | TECHNOLOGY REVIEWS, INCLUDING LTE, 5G, P25, NXDN, TETR

JOIN US FOR IWCE 2020

R, ETC. | FREQUENCY COORDINATION

ST

&

MEASUREMENT/COVERAGE

ANALYSIS

| PLANNING AND PREPARATION FOR COMMUNICATION SYST |

PRODUCT

CONFERENCE DISTRIBUTION/CHANNEL MARKETING

AND POLICY DISCUSSIONS | YOUNG PROFESSIONALS MENTORING |

Gain insights on industry trends through our five-day conference program, meet face-to-face with nearly 400 exhibitors in our 80,000 square foot Exhibit Hall. 6,500+ industry professionals from around the world attend from government, public safety, utilities, transportation, enterprise, professional services and the dealer community. ABOUT IWCE IWCE connects the critical communications ecosystem through education, networking and insights to help professionals leverage the evolving technologies to create a safer, more efficient and more interconnected world.

|

REGUL

March 30-April 3, 2020 COMUNIS WOMEN IN CRITICAL EXHIBITS April 1-2, 2020 Las Vegas Convention Center Las Vegas, NV

REGISTER BY FEBRUARY 21 Save up to $150 with our Advanced Prices! USE PROMO CODE AUVSI20 Save an ADDITIONAL 20% off select conference packages* and receive FREE access to our 80,000 square foot Exhibit Hall. IWCEEXPO.COM

*not valid on single-session tickets or already reduced government rates

In partnership with:

IWCEEXPO.COM #IWCE2020 JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 31

|


FEATURE

KNOCK KNOCK,

YOUR DRONE IS HERE: PACKAGE DELIVERY BY DRONES BECOMES REALITY By Jessica Reyes Sondgeroth A UPS Flight Forward drone prepares to lower a package to a customer. Photo: UPS

32

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020


T

UPS Flight Forward has been working with CVS Pharmacy for drone delivery of medicine. Photo: UPS

Drone package delivery is a reality in a few limited markets, but companies say this points the way to boosting business and cutting pollution in the future, once regulations permit large-scale operations.

he days of drones delivering goods to consumers across the United States are upon us, albeit modestly to start. And though 2020 seems poised to spawn yet wider distribution of drone delivery services, from medical prescriptions to fast-food hamburgers, it will take federal promulgation of final rules before the industry can officially cut the ribbon. With final federal rules governing drone operations in the pipeline, the U.S. Federal Aviation Administration (FAA) is, in the meantime, permitting exemptions to existing rules to enable the first class of commercial drone deliveries to infiltrate the nation’s growing door delivery service industry. Current FAA regulations prevent companies from flying drones beyond visual line of sight (BVLOS), over people, at night and with certain weight limits. However, the FAA in 2019 granted exemptions permitting limited commercial operations to a handful of companies, including Alphabet’s Wing and UPS subsidiary UPS Flight Forward. The hope, among regulators and drone companies alike, is that these steady developments will open doors to drones in a variety of marketplaces. “Regulators need to see innovative technologies like delivery drones in action because that helps them understand the issues that commercialization will present to the regulatory environment,” says UPS spokesperson Kyle Peterson.

A UPS worker prepares a Matternet drone for a UPS Flight Forward delivery. Photo: UPS

JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 33


A Flytrex worker prepares a drone for a delivery flight in Iceland, where the company has been operating such flights since 2017. Photo: Flytrex

Alphabet’s Wing spokesperson, Jonathan Bass, says Wing expects to share its key project findings publicly. Drone delivery providers estimate such delivery would cost less than traditional transportation methods, and those savings, combined with increased accessibility to products, would benefit businesses by increasing customer purchases. In a mature market, Wing estimates delivery by drone could increase sales in metropolitan areas by up to 27.4 percent per year for participating businesses. There are also environmental benefits. Assuming a high adoption rate across the metropolitan area, Wing estimates the drones replacing cars for delivery services could reduce carbon dioxide emissions by 40 tons per year. UPS subsidiary UPS Flight Forward was in September awarded the first full Part 135 Standard certification from the FAA to expand into the nations’ first commercialized drone airline. The certification means UPS can launch an unlimited number of drones for BVLOS, conduct flights at night and transport cargo weighing more than 55 pounds. 34

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

On Nov. 1, in partnership with drone logistics company Matternet, UPS Flight Forward delivered by drone medication from a pharmacy in North Carolina to a residential home, followed by a second delivery to a nearby retirement community. A remote operator monitored the autonomous delivery by the Matternet M2 drone as it hovered 20 feet above the property while lowering the package to the ground with a tether. The authorization expands on UPS’s work in March under a drone delivery pilot program at WakeMed Hospital in North Carolina, which executed more than 1,500 medical deliveries. Under the FAA and Department of Transportation’s Unmanned Aircraft System Integration Pilot Program (IPP), the UPS test project at WakeMed “helped develop some familiarity and confidence between the regulators and the UPS Flight Forward operation,” Peterson says. No doubt helpful in achieving its FAA certification, UPS, through its subsidiary UPS Airlines, brings with it a wealth of experience in aviation regulations.


IPP offshoots Launched in October 2017, the highly competitive Unmanned Aircraft System IPP includes nine regional participants, including state transportation agencies, airports and universities, each of which partnered with a variety of service providers, including AT&T, CNN, Apple, Alphabet’s Wing, Matternet and Flirtey and Flytrex, to explore different drone operations. Participants were first selected for the three-year program in May 2018. From that, a handful have moved on to attain other types of FAA exemptions. Wing is another one of those companies. Wing, an Alphabet subsidiary and Google offshoot, in April 2019 received an FAA Part 135 single pilot air carrier certificate. Wing has used the certification to launch a drone delivery pilot program in Virginia, bringing over-the-counter medicines and other healthrelated items from CVS Pharmacy. Before the partial certification, Wing partnered with Virginia Tech under the FAA pilot program to facilitate package delivery in rural and urban settings, while studying drone technologies such as detect and avoid, identification and tracking, radar systems and mapping tools. Nevada-based Flirtey also received partial FAA approvals after participating in the FAA’s pilot program. Flirtey is today conducting routine drone delivery demonstrations in suburban Reno, Nevada, and plans to scale commercial drone deliveries in 2020 beyond just healthcare. Flirtey and the City of Reno have partnered with a local emergency medical services provider to deliver automated external defibrillators to patients in need, as well as with NASA to conduct deliveries of medicine to rural healthcare clinics. The Flirtey Eagle drone delivery system enables one operator to oversee up to 10 delivery drones, each conducting 10-minute deliveries, boasting aircraft that can operate in 95 percent of wind and weather conditions.

Flying hamburgers Not all the FAA pilot programs are primarily focused on medical care. Among several partners in the North Carolina Department of Transportation’s FAA pilot program, Israel-based drone delivery company Flytrex will provide food delivery service to suburban areas, operating autonomous BVLOS flights. Flytrex boasts it has more experience, having deployed its drone delivery service in Iceland in August 2017. Flytrex CEO Yariv Bash tells Unmanned Systems that unlike its more high-tech, heavy duty drone delivery services offered by his competitors, Flytrex is targeting high bandwidth dispatch at the lowest cost. “I want to deliver your cheeseburger,” Bash says. That means simplifying drone delivery for the most economic value. Flytrex uses a hexacopter controlled by a human pilot and carrying up to 6.6 pounds, or up to eight hamburgers, for a distance of 3.5 miles and back and using a tether to lower the package from up to 80 feet. The drones do not fly in the rain and wind gusts stronger than 18 miles per hour. To be cost-efficient, low-margin payloads like burgers mean simpler logistical platforms than other drone delivery designs. With its vast range of

A Flytrex drone releases its payload via tether, which keeps the drone, and its attendant noise, well above the ground. Photo: Flytrex

deliverable products, Amazon in June unveiled its latest Prime Air drone design, consisting of “fully electric drones that can fly up to 15 miles and deliver packages under five pounds to customers in less than 30 minutes.” Though Amazon has not yet been authorized for commercial operation, it claims its “hybrid design” enables vertical takeoffs and landings, like a helicopter, while easily transitioning to and from vertical mode to airplane mode. It also comes equipped with technology to enhance the drone’s situational awareness. “Some drones are autonomous but not able to react to the unexpected, relying simply on communications systems for situational awareness,” the company says in a blog post. “If our drone’s flight environment changes, or the drone‘s mission commands it to come into contact with an object that wasn’t there previously — it will refuse to do so — it is independently safe.”

Safety, privacy, noise Among various differences in logistical operations, payloads, size and scope that various concerns about drone deliveries rise to the surface of the ongoing public debate over noise, security and privacy concerns over drones. The reason some delivery drones use a tether to lower packages to the ground is to reduce noise pollution and increase safety. The tether helps ensure the drone remains far enough away that it is hardly audible and securely out of reach of various ground-level interference. The Flirtey Eagle hovers at a height above trees and power lines, lowering the JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 35


package with the tether until it is delivered, then retracts the tether before returning back to the Flirtey Portal. “Flirtey has built and patented this safety and anti-tampering system so that if someone pulled on the tether, the tether has the ability to detach from the drone, ensuring safety,” says Flirtey spokesperson Daniela Urso. While UPS Flight Forward plans to develop a ground-based, detect-and-avoid system with a centralized operations control center to help it command BVLOS drone flights, various drone delivery providers are currently using unmanned aircraft traffic management software to ensure optimal routing, collision avoidance and to target a precise delivery location, although some are more equipped than others with sensors and cameras. That raises another public concern: privacy. While some delivery drones are equipped with cameras, others are not. Flytrex’s low-cost drone delivering your cheeseburger does not come equipped with a camera, while Wing’s drones do, although Wing maintains its capabilities are limited. “Wing aircraft carry a still-image, low-resolution camera used exclusively for back-up navigation in the event GPS is unavailable,” Bass says. “The safety camera points downwards and cannot be aimed towards particular objects. At no time is there a live feed of photos from the camera to anywhere or anyone, including the safety pilot supervising flights. In practice, people who do not use our service will not be recognizable in a photograph while our drone flies to its delivery destination.” And when it comes to noise — although most drones are not much louder than normal atmospheric noise — different drone companies have different solutions to reduce noise pollution. Wing, for example, “recently developed new propellers that emit a quieter, lower-pitched sound that blends in better with other sounds in a residential neighborhood.”

UPS says it is “constantly striving to improve the quality of its service while ensuring that it does not negatively affect the community. Noise issues are part of this equation, and UPS will continue to refine its equipment and operations to minimize the impact of noise on the public.”

Flirtey is conducting deliveries in Reno, Nevada, and plans to scale its operations in 2020 beyond its current healthcarerelated deliveries. Photo: Flirtey

Wing is now delivering packages via drone for residents in the Christiansburg, Virginia, area. Photo: Wing 36

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020


An invaluable resource for innovation, product development and R&D.

Subscribe today for access to: • Information on 5,250+ platforms associated with 1,750+ global organizations.

AUVSI’s Unmanned Systems and Robotics Database is the largest, most comprehensive and searchable database of unmanned vehicles operating in the air, ground and maritime domains.

roboticsdatabase.auvsi.org

• A fully searchable database containing 200+ parameters— size, performance, power, etc.—across all domains. • Detailed descriptions and specifications for integrated components and subsystems. • Database updates applied daily. • Regular emails with new and updated platforms, manufacturers and other information. • Exportable information—available in Excel and PDF formats.


FEATURE

‘CUSP OF A NEW ERA’

AUTONOMOUS SHIPS SET TO REVOLUTIONIZE MARITIME INDUSTRY By Rich Tuttle

The Maersk Evora docked at Pier 400 in the Port of Los Angeles. Photo: Port of Los Angeles

Autonomous ships could change the maritime industry as much as containers did when they were introduced some 50 years ago, according to a new report from shipping entities. Containers revolutionized trade and may have given globalization its biggest push, says “Smart Ships and the Changing Ecosystem” by SmartPort, a joint venture of Dutch entities based in Rotterdam. Today, it says, “the maritime industry is again on the cusp of a new era — one driven by increased digitalization and innovation, in particular, autonomous ships.” 38

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

The coming introduction of such ships, it says, “has the potential to impact all aspects of operations and business in the industry.” It’s a big industry. Global seaborne trade was valued at about $12 trillion in 2017, according to Statista, a provider of market and consumer data based in Hamburg, Germany. About 60 percent was attributed to container trade. Containers carried 1.83 billion metric tons of in 2017. The potentially attractive business case for autonomous ships — lower fuel consumption, fewer idle hours, fewer


personnel, and lower costs — has caught the eye of big container ship companies like Denmark’s A.P. Moller-Maersk, the Swiss-Italian Mediterranean Shipping Co., China Ocean Shipping Co., France’s CMA CGM and Germany’s Hapag-Lloyd. Norway’s Kongsberg is among the first tech companies hoping to capitalize on such potential. It aimed to boost its capabilities in maritime autonomy by acquiring Rolls-Royce’s Commercial Maritime unit in April of 2019. It’s also working with Yara International, a Norwegian producer of fertilizer, to develop a prototype autonomous container ship named Yara Birkeland. Yara has used trucks to move its products to Norwegian ports and wanted a cheaper alternative. Yara Birkeland would take the place of trucks by sailing in Norwegian waters. Kongsberg is handling the ship’s sensors and integration, as well as the electric drive, battery and propulsion control systems. The 260-foot-long ship is slated to go from manned operations in 2020 to fully autonomous operations in 2022. The Rolls-Royce deal, says Kongsberg CEO Geir Haoy, will help the company “lead the development of the future maritime industry globally.” Rolls-Royce Commercial Maritime’ expertise, he said, increases Kongsberg’s “leadership in maritime digitalization, ship intelligence and enabling technologies for autonomous vessels.” Some are skeptical of widespread use of commercial autonomous vessels. “The world will never see fully autonomous transoceanic commercial cargo ships,” writes Cmdr. David Dubay of the U.S. Coast Guard. “Once we get past the promises and hyperbole, the risk of collisions, legal liabilities and environmental calamity will ensure that some critical number of humans will persist on board ships,” he says in a June 25, 2019, article for the Center for International Maritime Security. “Advances in technology will continue to make shipping safer and more efficient, but they will not eventually replace the human masters and crews that serve on today’s commercial vessels.” He received return fire. “If self-driving cars are to come to pass, then autonomous or remote-controlled ships are a trivial matter,” because of slower speeds and less demanding requirements for position updates, one reader comments. Another writes, “Redesigning ships without any hotel services (fresh water, greywater, sewage, HVAC, messing facilities, food/stores storage, cleaning/laundry, human safety, etc.) marks a significant decline in complexity and maintenance of auxiliary services, and an ability to focus design solely around cargo carriage.” Yet another writes it is “a bit short-sighted to assume that today’s challenges are not insurmountable, or that business models will not evolve over time ... it may take a lot longer than many overly optimistic interests frequently portray, but unless we’re talking about breaking laws of physics, I certainly would avoid the word ‘never’!” Today, says Capt. Marc Deglinnocenti, whose time at sea dates to 1974, “there are some instances where 100 percent autonomous vessels are not only appropriate, but safer than manned vessels.” Deglinnocenti, who also is a maritime technical writer, says in an email that such vessels might work in “areas of extremely inherently dangerous operations, such as mine sweeping and ordnance removal, scientific vessels studying volcanic islands forming, and fire boats inserting themselves directly to flames and toxic fumes. ...”

An aerial view of the Port of Los Angeles’ side of the San Pedro Bay Port Complex. Photo: Port of Los Angeles

Autonomous shipping on oceans and rivers could revolutionize the maritime industry, but technical challenges remain and there is skepticism about going fully autonomous within the industry.

JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 39


Timelines The first fully autonomous ships could begin to appear in such roles “in less than five years,” the SmartPort paper says. In 10 to 30 years, it says, fully autonomous shipping lines will be in business. In 50 years, three or four big container-ship companies will have survived multiple overcapacity and consolidation cycles, says a 2017 report by the Londonbased management consulting firm McKinsey & Co. These companies will be “digitally enabled independents with a strong customer orientation and innovative commercial practices, or small subsidiaries of tech giants blending the digital and the physical,” it says. “All winners, closely connected through data ecosystems, will have fully digitized customer interactions and operating systems.” Currently, the SmartPort paper says, “most shipping companies are not seriously investing in autonomous technology.” The question is, “are we waiting for a new entrant like Tesla, or even an outsider like Amazon to shake up and disrupt this market?”

Guidelines and examples Meanwhile, the United Nations’ International Maritime Organization, based in London, has approved an initial set of guidelines for trials of autonomous ships. IMO, charged with regulating worldwide shipping, says the guidelines emphasize the importance of “safety, security and protection of the environment.” Some say autonomous merchant vessels will ultimately become more numerous than smaller vessels because there are simply more merchant ships. For now, however, Deglinnocenti is “against 100 percent merchant vessels” for several reasons, including “safety, security, laws, ethics, and jobs.” Nor does he believe that artificial intelligence systems Switzerland's ABB and Keppel of Singapore are rerofitting a tugboat for autonomous operations. Image: ABB

now being developed for ship maneuvering are totally reliable. “I do value [artificial intelligence maneuvering] systems for what they can deliver to the professional mariner, but I see AIM as a tool to aid the mariner and help with workforce shortages. I don’t see AIM as a complete replacement of the mariner.” The period leading up to fully autonomous ships will be “particularly challenging” because they “will need to operate in mixed environments alongside analog ships, boats and traditional infrastructure,” according to the SmartPort paper. Ports like Singapore and Rotterdam are preparing for increased autonomy, as is the largest container port in the United States, the Port of Los Angeles. Fearing for their jobs, thousands of International Longshore and Warehouse Union workers have organized to oppose automation at Los Angeles and Long Beach. Work also is underway to automate tugboats that ply port waters. ABB, based in Zurich, Switzerland, is working with Keppel Marine and Deepwater Technology of Singapore to retrofit a harbor tug with digital systems. Their goal is to begin autonomous operations in the port of Singapore in late 2020. The vessel is expected by the companies to be South Asia’s first autonomous tug. Airbus and Norway’s Wilhelmsen Ships Services have begun testing in Singapore of an aerial drone designed to carry small, time-critical packages from the shore to ships in port. Ports and ocean-going ships are two of the three major categories of the maritime industry outlined by the SmartPort paper. The third is inland, or barge, shipping, in which vessels move between ports and destinations on local rivers.

Rolling on the river Business is big on the rivers. In 2017, more than 13,000 inland vessels were registered to operate in the Rhine and Danube basins, and 300 operate each day on the lower Rhine, according to Benjamin Boyer, administrator of the Central Commission for the Navigation of the Rhine, based in Strasbourg, France. In the United States, some 600 million tons of cargo move each year on the 12,000 miles of navigable inland waterways like the Mississippi and Ohio. “About a third of that is coal, close to another third of it is petroleum products, and maybe almost as much is grain and wheat products,” says Steve Little, president of Crounse Corp., in Paducah, Kentucky. Crounse, one of a number of American river boat companies, moves about 30 million tons of cargo a year. It has over 1,100 barges and 35 tow boats. “We decided to locate in Paducah

40

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020


because Paducah [on the Ohio River] is the hub of the nation’s inland waterways, geographically, and it’s the perfect location” from which to operate barges,” Little says in a YouTube video. As in the ocean cargo-shipping business, there’s skepticism about autonomy in the river boat industry. The fact that expensive simulators are used to train river boat pilots is an indication that autonomy in that business isn’t just around the corner. The stakes are high, and today, at least, a sure human hand at the controls is a must for river boat companies. A single barge equals about 70 18-wheelers, and six lashed together — a relatively small tow — equals about 420 18-wheelers. A typical river tow might consist of 35 to 42 barges. The speed is slow, but the physics of momentum mean there’s no such thing as a small collision. Then there are the rivers themselves. Water depth changes, shoals can come and go, there are tides and currents and storms, spring floods and winter ice. Even so, as technology is increasingly felt, the boats themselves may initially be smarter than the shore-based tech infrastructure of sensors that will have to support them, says Joe Celano of San Diego-based Trabus Technologies, which advises industry and government on wireless technologies, cybersecurity and artificial intelligence. Part of the problem, he says in an interview, is that while many government agencies have their hand in the world of American rivers, there’s no one source for information on, say, water depth. Better sharing of information would cut wait times at locks

and terminals and improve the timeliness and accuracy of navigation information, among other things, says Celano, who leads Trabus’ efforts in the development of big data analytics for U.S. inland waterways. Celano, who is also U.S. representative on the Smart Shipping Working Group at PIANC, the World Association for Waterborne Transport Infrastructure, based in Brussels, Belgium, acknowledges the vulnerability of future autonomous vessels to cyber attack. The greater the level of interoperability, the greater the exposure to cyber attack, and the marine industry has long way to go when it comes to protection against such attacks, an industry official says. But, he adds, other industries are working the problem too. “A lot of lessons to be learned,” he says.

31.31mm

26mm

MicroLink

Trusted by Pilots. Innovated for Drones. The Smallest BVLOS C2 Data Link Built to Aviation Standards Designed for Precision - Achieved Through Experience Learn More at uAvionix.com/MLPrecision

JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 41


CHAPTER NEWS DC Capital The DC Capital Chapter held an expert panel discussion on autonomous ground vehicles at an event in October in Arlington, Virginia. Panelists were Kevin Dopart of the U.S. Department of Transportation’s Intelligent Transportation Systems Joint Project Office; Reed Young and Dave Handelman, both of the Johns Hopkins University Applied Physics Lab; Jessica E. Rojkowski of MITRE Corp.; and industry veteran Robert Finkelstein, of Robotic Technology Inc. Finkelstein said autonomous ground vehicles, including self-driving cars, will be “disruptive and transformative,” and called for a new level of autonomy beyond the familiar SAE-derived five levels: A level 6, where autonomous vehicles could access information from the cloud to help them learn and even converse with people, so “a vehicle is not just a vehicle, but a team member in a squad or a platoon.” Rojkowski said future vehicles likely won’t be individually owned, but deployed in fleets that will operate all the time instead of spending most of their time being still. They will be electric and gas stations will go away, which will have ripple effects on the economy, as snacks and other goods sold at gas stations will go away, too. Dopart said regulations are slow to adjust and it will take “many years” to change them to put self-driving cars on the road, but Finkelstein predicted, “at some point, it will be illegal to drive a car where you can’t ride a horse.” Handleman said future cars might come with personalities or assistants that could follow their owners around and then interact with the vehicle. Consumers might purchase a vehicle based on its personality. “Think Siri, think Alexa, but it moves,” he said.

Arizona; it was the place to be for UAS professionals in Arizona that heard from a star lineup including Eric Bergesen from UPS Flight Forward; Jeremy Grogan from the Federal Aviation Administration; Paul Brierley from the University of Arizona’s Yuma Center of Excellence for Desert Agriculture; Romeo Durscher from DJI; Sami Mian from NASA; and many more. Live demonstrations, workshops and demos showcased what is next in the future of UAS.

Speakers at the Unmanned Arizona Summit during National Drone Safety Awareness Week. Photo: Saguaro Chapter

Atlanta The Atlanta Chapter of AUVSI put on a great event of fun recreational flying in South Atlanta. Kim Schofield of the Georgia House of Representatives joined us to learn about aerial technology and took time to fly a drone herself. There were demonstrations of thermal imaging, noiseless propellers and 3-D printed components for drones. Many kids took part to fly for the first time and learn about safety working with drones. For the brave of heart there was even a racing course for FPV flying. Everyone was a winner. Another event is planned for April 2020!

MITRE Corp.’s Jessica E. Rojkowski speaks at the chapter event on autonomous ground vehicles. Photo: AUVSI

Saguaro The AUVSI Saguaro Chapter participated in the Unmanned Arizona Summit that took place during the FAA National Drone Safety Awareness Week. The event proved to be the biggest and best yet for

42

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

Even young fliers were able to join the fun at the Atlanta Chapter event. Photo: Atlanta Chapter


OUR CHAPTERS: ATLANTA

BIG SKY

CASCADE

CHANNEL ISLANDS

COASTAL PLAINS

D.C. CAPITAL

EMERALD COAST

EMPIRE STATE

FLORIDA PENINSULA

GATEWAY

GREAT LAKES

GREAT PLAINS

HAMPTON ROADS

HEARTLAND

INDIANA

ISRAEL

LIBERTY

LONE STAR

MISSISSIPPI

MOUNTAIN WEST

NEVADA

NEW ENGLAND

NORTH CAROLINA

PATHFINDER

LOUISIANA PELICAN

RIDGE AND VALLEY

ROCKY MOUNTAIN

SAGUARO

SAN DIEGO LINDBERGH

SEAFARER

SILICON VALLEY

SINGAPORE

TWIN CITIES

ESPAÑA • SPAIN

UK

USA-OK

JOIN AUVSI For information on joining a chapter, contact: Amanda Bernhardt, Chapter Relations Manager abernhardt@auvsi.org To visit local chapter websites, scan this QR code or visit auvsi.org/MembershipandChapters/Chapters

>

JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

| 43


FAR OUT

MIT HELPS AUTOMATED VEHICLES SEE AROUND CORNERS By Brett Davis

MIT researchers have developed a system that can sense tiny changes in shadows on the ground to help autonomous systems determine if a moving object is coming around a corner. This work, funded by the Toyota Research Institute, could one day help keep self-driving cars from hitting other cars or pedestrians or help guide robots through hospital hallways as they deliver medicine or supplies. MIT researchers conducted experiments with an autonomous vehicle driving around a parking garage and a self-driving wheelchair moving through hallways. In a paper presented at the International Conference on Intelligent Robots and Systems, they said the new system bested traditional lidar systems by more than half a second — not a lot, but enough to forestall accidents. The system, dubbed ShadowCam, uses sequences of video frames from a camera aimed at a particular area, such as the floor in front of a corner. Changes in light intensity can indicate whether an object is stationary or moving. An earlier version relied on simplified QR codes placed on a floor; to be used in dynamic environments, the latest version combines images registration — which overlaps images to compare for changes — and visual odometry, which estimates the motion of a camera in real time and which has been used on Mars Rovers. 44

| UNMANNED SYSTEMS | JANUARY - FEBRUARY 2020

As ShadowCam moves, it uses the registration method to overlay images from the same viewpoint as the robot, zeroing in on the same patch of pixels where a shadow is located, seeking any subtle changes between images. So far, the system has only been tested in indoor settings, where speeds are lower and lighting conditions are more consistent. Researchers plan to develop the system to work outdoors as well, and to speed up the ShadowCam’s shadow detection and automated the process of selecting areas to search for shadows. “For applications where robots are moving around environments with other moving objects or people, our method can give the robot an early warning that somebody is coming around the corner, so the vehicle can slow down, adapt its path, and prepare in advance to avoid a collision,” says paper coauthor Daniela Rus, director of the Computer Science and Artificial Intelligence Laboratory. “The big dream is to provide ‘X-ray vision’ of sorts to vehicles moving fast on the streets.” Next, the researchers are developing the system further to work in different indoor and outdoor lighting conditions. In the future, there could also be ways to speed up the system’s shadow detection and automate the process of annotating targeted areas for shadow sensing.


SOCIAL UPDATE

TRENDING

FOLLOW US ON SOCIAL MEDIA

facebook/ auvsi instagram/ auvsi twitter/ auvsi JANUARY - FEBRUARY 2020 | UNMANNED SYSTEMS

45

| UNMANNED SYSTEMS | JUNE NOVEMBER-DECEMBER 2019 2019

| 45


get CONNECTED INFORMED INVOLVED

AUVSI membership allows you to contribute to the unified voice of the unmanned systems industry as we advocate for policies and regulations that encourage industry growth and innovation.

Learn more at auvsi.org/why-join-auvsi

Profile for AUVSI Unmanned Systems Magazine

AUVSI's Unmanned Systems magazine — Jan.-Feb. 2020