While this may seem a logical progression, the path is fraught with peril. In particular, the boundaries between Levels 2, 3, and 4 are at least fuzzy, and following this path to Level 4 presents many challenges that are extremely hard to face when one takes into consideration not only the technology, but also the fundamental characteristics of human nature. Indeed, any automation system that, by design, relies on a human supervisorâ€™s intervention in certain situations implicitly assumes that the human supervisor is part of the system. Unfortunately, no matter how well the automation part is designed, No matter how well the human nature cannot be designed to specifications, especially automation part is designed, when one needs to address the general public versus highly trained professionals. human nature cannot be
designed to specifications.
For example, even though the Tesla Autopilot is technically described as Level 2 automation, requiring constant attention by the driver, its undeniable sophistication, as well as the hype surrounding the technology, make it easy to erroneously think of it as Level 3, allowing the driver to cede control completely. While we await the results of the NHTSA investigation, it seems likely that one of the causes of the Tesla fatal accident was the confusion between what the Autopilot could provide (Level 2), and what the driver expected (Level 3). Furthermore, even the most disciplined driver would not be able to constantly remain alert and maintain situational awareness for more than a few minutes at a time when the car is under control of an automated system. This is a well-known and hard-learned fact in the aerospace community: many airliner crashes, killing thousands of people, have been traced back to issues arising at the interface between humans and automation, such as mode confusion, loss of situational awareness, and reduced ability to handle emergency situations. Paradoxically, the more sophisticated an automation system is, the more it can lead a human pilot to lose the ability to intervene effectively. It is shocking that at least part of the automotive industry seems to be unaware of this body of knowledge.
Published on Nov 18, 2016
Annual magazine review of MIT Aeronautics and Astronautics Department research and educational initiatives.