Discussion paper by the SATW topical platform on autonomous mobility
How safe should automated vehicles be?
Authors/group of experts: Wolfgang Kröger (main author, member of SATW, head of the Autonomous Mobility Topical Platform, emer. prof. ETH), Andreas Burgener and Thomas Rücker (autoschweiz), Bernhard Gerster (Bern University of Applied Sciences), Stefan Huonder (FEDRO), Thomas Küchler (SOB), Manuel Kugler (SATW), Marco Laumanns (ZFF), Jürg Michel (BLS, formerly PostBus), Thomas Probst (University of Fribourg & SAAM), Reto Schneider (Amstein+Walthert, formerly SwissRe)
Project management and editing: Christian Holzner (SATW), Stefan Scheidegger (SATW)
November 2024
Abstract
Automated vehicles promise to greatly reduce the influence of humans as the direct or indirect cause of serious accidents and critical driving situations, thereby further increasing road safety. They do this by transferring control and driving functions from humans to a highly sophisticated technical system. Automated vehicles can also be expected to bring benefits such as increased driving comfort and improved traffic flow.
Optimists anticipate highly automated cars "for everyone" and the start of market penetration around 2030. Fully automated vehicles, "all-rounders", will probably not follow until much later.
The procedure from development through various tests to certification has not yet been established [Yurtsever et al. 2020]. In the concept of "type approval", which continues to be pursued in Europe, the manufacturer is required to prove to the authorities that the vehicle fulfils the safety requirements.
Safety is often understood as the absence of an unacceptable risk of accidents. The risk of automated vehicles must not be higher than that of conventional, human-driven vehicles; the risk balance should be positive. This qualitative target is vague and, in our opinion, needs to be made more specific. The answer to the central question "How safe is safe enough?" should not be limited to the purely technical and mathematical but should also include the social and cultural context.
The SATW working group's proposal is to be understood as assistance from a neutral, scientific body. It aims to support the development and introduction of automated vehicles and, by means of quantitative targets, enable an objective assessment of the progress made in terms of safety and help to create a common state of the art.
The proposal is based on existing regulations, standards and proposals for autonomous vehicles and from other areas of comparable benefit. Accordingly, an accident risk for persons within the automated vehicle is considered unreasonable if the value of 3x10-8 fatalities per operating hour or 4x10-7 fatalities and serious injuries per operating hour is exceeded. Ten times lower values are proposed for reasonable accident risks, i.e. a maximum of 3x10-9 fatalities, or a maximum of 4x10-8 fatalities and serious injuries per operating hour. Cost-benefit considerations should be permitted between the two threshold values.
The SATW Autonomous Mobility Topical Platform provides clarity regarding the development stages and associated technologies, analyses their development status, highlights key aspects, including opportunities and risks, and develops suitable solutions in the event of problems. The overarching aim is to provide an expert and fact-based analysis of the complex topic, while maintaining the necessary neutrality, to raise general awareness, trigger initiatives where necessary and counter undesirable developments as early as possible. To date, the focus has been on automated individual vehicles as enablers of future networked mobility.
1 Introduction
The basic idea and development goal of automated vehicles (AVs) is to transfer control and driving functions from humans to a highly reliable technical system. This combines sophisticated, largely innovative software and hardware using mechatronics, digitalisation and artificial intelligence (AI) methods. AVs promise to partially or completely relieve the driver of the dynamic driving task and thus increase driving comfort, improve traffic flow and make mobility more efficient [acatech 2018]. Paving the way are modern assistance systems, for example for lane keeping or distance keeping, which are already installed in many cars or are already mandatory for new cars.
One driver of a further increase in the degree of automation is the increase in road safety, as the influence of humans as the clearly dominant cause of serious accidents and critical driving situations could be reduced even further. However, the benefits will only begin to materialise when many vehicles are at least highly automated and penetrate traffic. These are vehicles that can drive autonomously in specified areas and under restrictive conditions (Operational Design Domain, ODD) (according to SAE level 4). In the case of full automation (SAE level 5), these restrictions no longer apply, and the driving task is performed entirely without human assistance or intervention under all normal circumstances; no person needs to be present (see Fig. 1).
2021
The mandatory prerequisite is the technical maturity and proof of the safety of the AV as part of the approval process: the sensor systems must be able to reliably localise the vehicle (with or even without external assistance) and reliably perceive the constantly changing environment, including any obstacles that may suddenly appear, and comply with the applicable traffic regulations (including speed limits, right of way situations). The high-performance computer system in the AV must
process enormous amounts of generated data1 and derive decisions and control commands from it. All of this must be done in real time, as independently as possible of a connection to the manufacturer and intelligent infrastructure elements.
Conditionally automated vehicles of SAE level 3 are already available today and some are already in operation abroad; in Switzerland, this will be possible from 2025 with the ordinance on autonomous driving. Special vehicles of the highly automated SAE level 4 are also in (partly commercial) test operation, such as fleets of "robotaxis" in San Francisco. Optimists expect SAE level 4 passenger cars to be affordable for a broad customer segment and to begin to penetrate the market around 2030; fully automated SAE level 5 vehicles will probably not follow until the later 2040s - if at all [ERTRAC 2022]. The introduction will therefore be gradual, from the automated handling of simple to more complex driving conditions and situations.
According to the authors, the entire procedure from the basic architecture and development to readiness for construction, testing and improvement through to official certification is not yet sufficiently established. For Europe as a party to the "Vienna Agreement"2, the type approval procedure practised to date is being established. Within an audited multi-stage process, the manufacturer must prove that the AV fulfils the safety requirements. This is understood to mean the "absence of real and potential conditions that could cause harm or death to persons, whether inside or in the immediate vicinity of an automated vehicle, following its design basis loss of function/dangerous failure". According to [EU 2022/1426], the risk is considered unreasonable if it is higher than the risk of a human-driven vehicle of the same purpose in a comparable situation and/or is below a socially accepted threshold [ISO 2626].
The visionary goal of zero risk ("Vision Zero") often appears at a high level of abstraction. This is less a realistically achievable target and more a demand to continuously increase the safety of AV through technological developments and their adoption.
"Safety" is of crucial importance for the development, authorisation and acceptance of automated vehicles. This is not a purely technical-mathematical parameter but includes the technological and socio-cultural context more comprehensively. This raises the familiar question from other areas: "How safe is safe enough?" The answer must include various aspects and factors and consider the interaction between vehicles with various levels of automation, the occupants and other vulnerable road users. An answer to the desired level of safety from a societal perspective has yet to be found.
This document is an attempt to narrow down such an answer and thus contribute to the creation of a common state of the art. It is based on knowledge acquired as part of the SATW thematic platform [SATW 2022], together with a comprehensive review of current approaches and solutions. The document is limited to the individual AV as a central element of autonomous mobility and initially disregards the requirements for the necessary infrastructure (traffic guidance and signalling systems, control centres for remote control in emergencies, assistance, etc.). It is divided into a
1 According to the ADAC, each vehicle needs around 5 GB/min, and the computing power required is equivalent to around 15 modern laptops.
2 The UN Convention on Road Traffic was signed in Vienna in 1968 and has been modified since 2015; it has permitted automated driving at SAE levels 3 and 4 since 2021, although level 5 is only permitted for test purposes with special authorisation.
more conclusive main section and two annexes that substantiate the statements made in the main section. Disruptions to the flow of traffic due to technical defects or insufficient availability of communication channels and a possible resulting loss of confidence are not considered.
2 Compilation of existing assessment approaches and safety target values
2.1 Framework conditions
Automated vehicles (AVs) will probably only become a reality for the general public once authorisation procedures and the associated safety requirements and evidence have been established to a sufficient extent. Before this happens, a number of technical hurdles still need to be overcome, particularly in the area of software and hardware. In addition, a personal benefit for a large group of buyers or attractive business models for mobility providers must be evident. As innovative vehicle systems, also as part of future autonomous mobility, AVs must also meet with social acceptance. This will probably require targeted marketing efforts by the manufacturing industry. In the longer term, AVs are expected to increase road safety compared to current vehicles driven by people. In addition, there are hopes for an increase in comfort, harmonisation of traffic flow and optimisation of usage and parking space as well as new logistics concepts. These expectations are countered by scepticism and fears, which culminate in fears due to the assumption of control and driving functions by anonymous technology (including the associated algorithms), which must be countered with suitable measures to build trust.
The reference for the safety to be guaranteed is the modern car (equipped with assistance systems) driven by humans and the current traffic and accident situation. The current frequency of serious accidents should be undercut, and "unreasonable risks" should be avoided. This social requirement for safety has priority and must not be restricted by other ethical or economic considerations. On the other hand, residual risks that can be considered acceptable due to their "smallness" and in a social and moral context must be accepted. A German ethics commission calls for a positive risk balance and the avoidance of dilemma situations "by design"; the protection of people should take precedence over other considerations of benefits [BMVI 2017].
The term "risk" comprises - as is usually the case - the probability of occurrence of undesirable events multiplied by their consequences (extent of damage). With regard to automated driving, this means the risk includes the probability of accident situations and their health consequences for the people involved inside and outside the AV involved.
In addition to rather vague, qualitative safety requirements, there is an emerging realisation that quantitative safety targets are necessary; initial proposals for such values already exist. These are often formulated as individual value(s), such as the maximum permissible mortality risk. In some areas, quantitative safety targets are also formulated as a risk matrix; risk aversion is considered by giving greater weight to the consequences of accidents if the latter can be very high.
2.2 Comparison of target values
In addition to qualitatively semantic safety requirements, such as the required "absence of unacceptable risks", there are proposals for quantitative safety requirements for AV and in other
sectors for their concretisation and operationalisation. Most proposals/practices specify a target value for the individual mortality risk, either as the only value or as part of a broader range of losses, linked to associated probabilities of occurrence.
Target values for automated vehicles
According to the "Implementing Regulation" [EU 2022/1426, Annex II, 7.1.1], the EC proposes a permissible mortality risk of max. 10-7 per operating/travelling hour. This proposal is based on aggregated current EU accident data and applies to the market launch phase.
The relevant ISO standards [ISO 26262:2018, ISO 21448:2022] require "unreasonable risks to be avoided (not diluted or minimised)" and the fulfilment of quantitative targets. These include the maximum number of accidents per operating hour of a vehicle, as defined under MEM, GAMAB:
- A technical system should not contribute significantly to the existing MEM (minimum endogenous mortality), e.g. not be > 1/20th MEM, because people are exposed to the risks of several systems whose individual values are 10-6. A new system should be better than the old one, which could conservatively result in a value of 10 -7 per person, per year.
- According to GAMAB (globalement au moins aussi bon - generally at least as good), "a new system must promise an overall risk level that is at least as low as that of an equivalent existing system"; in Germany, for example, this leads to a mortality risk in road traffic of <10-5 per person, per year.
- Cost-benefit considerations are allowed for a transitional area between the acceptable and unacceptable risk, according to the principle "as low as reasonably practicable" ALARP. The following values are given for the ALARP regions: risks higher than about 10-4 per person, per year are unacceptable, risks less than about 10-7 per person, per year are widely accepted (see A.1.1).
- In addition, requirements of ASIL (Automotive Safety Integrity Level) classes must be met to demonstrate functional safety, albeit limited to assumed faults in electrical/electronic hardware systems, while other sources of faults are only recorded qualitatively. The assessment matrix (see Fig. A.1) shows, for example, an associated failure probability of < 10-8 per operating hour for "severity class B3" (most severe violations) / "controllability class C3/D" (difficult).
In anticipation of international requirements, the German ordinance regulates the operation of motor vehicles with automated and autonomous driving functions [BMDV 2022]. It requires the manufacturer to develop a safety concept for functional safety. The level of safety should be higher than that of vehicles driven by people The required state of the art is deemed to be met if the requirements of the above ISO standards and UNECE Regulation No. 155 on cyber security are fulfilled.
The German Ethics Commission mentioned above considers the authorisation of automated systems to be justifiable if they promise at least a reduction in damage in terms of a positive risk balance compared to human driving performance under comparable conditions.
Target figures for other areas
For chemical and other hazardous installations, the risk matrix of the Swiss Major Accidents Ordinance (MAO) has an anchor point to the unacceptable range with a tenth of the average
probability of death (10-3 per person, per year), the difference to the acceptable range is a factor of 100; damage is weighted more heavily than probabilities (aversion coefficient 2). For example, some 100 fatalities are assigned a probability per year, per plant of 10-7 (unacceptable) or 10-9 (acceptable).
For rail travellers, the general limit is ≤ 8.8x10-5 deaths per person, per year.
According to the draft of a tiered risk matrix of the Federal Office of Civil Aviation of Switzerland (FOCA), with a transitional area and risk aversion taken into account, a maximum probability of 10-7 per ground movement/flight hour is considered acceptable for aviation, for example, for events with 2 to 6 fatalities, and 10-5 as unacceptable.
Information on target values in the field of nuclear technology (e.g. core melt frequency calculated once in ten thousand or one hundred thousand years of operation) on socially accepted accident risks in Switzerland is summarised in Annexes A.2.2 and A.2.5.
Conclusion
It is generally difficult to compare the target values; one reason for this are different units. For example, values are given integrated over a year or in relation to a driving situation; assuming an annual mileage (e.g. 25,000 km) and average speed (e.g. 40 km/h). The units could be converted and checked for compatibility, however. Another reason are different reference systems such as general population risks, risks of other activities, accident statistics for conventional vehicles or apparently accepted risks. It also remains unclear how "fatalities" are defined, whether only acute fatalities really count or whether serious injuries with subsequent fatal consequences are included.
3 SATW’s proposal for AV in road traffic
The following explanations are to be understood as suggestions and assistance from a neutral scientific side and are intended to serve the creation of a common state of the art and the standardisation of safety requirements. They are based on the most comprehensive considerations possible.
The proposal3 assumes that the required or achieved and socially accepted safety in other areas with the same purpose, such as the railway and aviation transport systems, should be used for orientation. In addition, the comparative system should also transfer control to "others", often to an intelligent technical system. If the transport is deemed necessary, the associated accident risk can be considered to have been entered into involuntarily by the user, even if he or she is not necessarily aware of it. Accidents seem to be socially accepted as an inherent risk of the transport system.
Human life is regarded as the primary protected good, all people are equally important. It is assumed that protecting people also adequately covers the protection of other assets. Objects of protection primarily include the driver and other persons inside the automated vehicle (AV) as well as other vulnerable road users in the immediate vicinity of the vehicle.
3 Please refer to the material compiled in the appendices for detailed reasons.
It is assumed at this point that the process for the approval of AVs will adopt the principle of type approval [EU 2019/2144] that has been customary in Europe to date4 . The overarching aim of the process is to avoid accident risks arising from driving performance. AVs should also be considered reasonable in terms of safety compared to today's vehicles driven by people and in the technical, moral and social environment. This (semantic) objective is largely in line with international regulations and standards, but in the authors' opinion, is too vague and requires concretisation and operationalisation through quantitative targets.
The view is shared that the manufacturing industry is responsible and competent for defining the safety requirements - within the framework set by the legislator. The same applies to the evidence required for authorisation, including virtual and physical testing. Government agencies should limit themselves to monitoring and checking compliance with the regulations and specifications and ensuring adequate quality.
Knowing full well that "risk" is not a clearly defined term and always has a subjective component characterised by individual perception in addition to the objective, mathematically tangible component, the following considerations are limited to the "objective" risk commonly used in technology and the insurance industry, with the probability of occurrence of unwanted scenarios and their consequences as the determining elements. Both elements are multiplicatively linked and totalled, resulting in an overall risk figure as a measured variable. This can then be compared across different domains and areas. The aversion to events with extremely low probability but very high damage that is often brought into play is disregarded, as such events can be practically ruled out if appropriate protective measures, e.g. against cyber-attacks, are assumed.
As a quantitative target, we consider a single value to be more suitable than a risk curve or a risk matrix. As a target value, we recommend the aggregated fatality or mortality risk to which the accident participants are exposed if they use AV driving services within the defined operational design domain (ODD). A second, supplementary parameter also takes into account seriously injured persons among those involved in the accident.
Methods appear to be available to provide corresponding evidence, which is not the case for the evidence of reasonable risks in the larger social environment, considering all causes of risk. The validity of the evidence is reduced by the lack of reliable empirical data in the market launch phase but can be increased to a reliable level with increasing market penetration.
Threshold values can be derived from current national and international accident statistics; these represent the limit to the unacceptable accident risk. The requirement that the automated driving vehicle is at least as safe as a human-driven vehicle can be taken into account5 . As automated driving promises to have a positive impact on the accident rate and the risk balance due to the
4 It is questionable whether this can be limited to the vehicle and whether other necessary systemic components outside the vehicle, such as those for networking with other vehicles or the infrastructure, should be included in the authorisation.
5 It should be noted that these statistics do not provide any information on how many accidents were avoided because the driver neutralised the incorrect behaviour of other road users by reacting intuitively and thus prevented an accident. As soon as a vehicle steers itself, this safety-enhancing factor no longer applies.
elimination of human error sources, the value of the reasonable accident risk should be lower; according to our considerations by a factor of ten6 .
An evaluation of the Swiss accident statistics7 leads to an unacceptable accident risk for people in an AV travelling on public roads if the value of 3x10-8 fatalities or the value of 4x10-7 fatalities and serious injuries, in each case per operating hour, is exceeded. The proposed values for reasonable risks are an order of magnitude lower, i.e. a maximum of 3x10-9 fatalities, or a maximum of 4x10-8 fatalities and serious injuries per operating hour. The ALARP principle should be applied in the area between the threshold values for reasonable and unreasonable risks8
These aggregated values are to be understood as guidelines and were derived for Switzerland. Together with the logic developed, they can serve as a guide for other European countries but may need to be adapted using specific accident statistics. They could apply to the introductory phase of passenger cars and other SAE level 4 AVs with associated ODD, are to be reviewed on an ongoing basis using specific, centralised accident monitoring and could change over time with increasing operational experience and an improved database. A breakdown by vehicle type (in addition to different types of cars, also by other forms of AV use) and by ice-age areas (such as expressways/motorways, rural or urban areas) would be desirable, but is not reliably possible based on the current data situation.
Communication sovereignty and responsibility for communication strategies should lie with the state or state-authorised bodies, as should the responsibility for dealing with "incidents", in particular accidents resulting in fatalities and serious injuries and overlooked hazards that are normally managed by people without any problems. A clear allocation of roles to demonstrate the safety of authorised, operational AVs and to ensure their safety in the overall system is essential and must be developed quickly, along with the associated forms and processes of participation.
Ethical considerations must be incorporated into the guidelines for the design (e.g. avoidance of dilemma situations) and programming of algorithms (e.g. prioritisation). The requirements for new automated systems to positively influence the risk balance must not be thwarted by ethical considerations; the most important overall objective is to achieve the most positive risk balance possible.
References
acatech (2018): Karsten Lemmer (ed.) Studie neue autoMobilität II (in german). / https://www.acatech.de/publikation/neue-automobilitaet-ii
6 This factor takes account of the expected reduction in the risk of accidents if human causes are eliminated and is in line with approaches in other areas/countries.
7 According to the 2023 microcensus conducted by the Federal Statistical Office (FSO), the annual average probability of a person being seriously injured or killed in a passenger car in Switzerland between 2019 and 2021 was around 3x10-7 per hour of exposure, and slightly more than a factor of 10 lower for fatalities alone. For the railway, the highest value (2021) for seriously injured occupants was 3x10-8 per hour of exposure; there were no fatalities. The socially relevant probabilities of people being seriously injured or even killed in and by a nearby vehicle were close to 10-6 for cars and almost 2x105 per hour of exposure for trains. The statistics for trams are similar to those for trains, while the figures for different types of buses are similar to those for cars.
8 In other words, risk-reducing measures are required if costs "only" increase proportionally.
FEDRO (2023): Documents for the consultation on autonomous driving. In: https://www.astra.admin.ch/astra/de/home/themen/intelligente-mobilitaet/rechtliche-situation/vernehmlassung-verordnung-automatisiertes-fahren.html
Implementing Regulation (EU) 2022/1426 laying down a rule for the application of regulation (EU) 2019/2144 of the European Parliament and Council as regards uniform procedures and technical specifications for the type-approval of the automated driving system (ADS) of fully automated motor vehicles, 5 August 2022
Federal Ministry for Digital and Transport Affairs (BMDV), Ordinance regulating the operation of motor vehicles with automated and autonomous driving functions, 86/22, Feb. 2022
Federal Ministry of Transport and Infrastructure (BMVI), Report of the Ethics Commission on Automated and Connected Driving, June 2017
ERTRAC, Connected, Cooperative and Automated Mobility Roadmap, 18 February 2022. https://www.connectedautomateddriving.eu/wp-content/uploads/2023/06/Connected-Cooperativeand-Automated-Mobility-RoadmapV10_18022022_ERTRAC.pdf
ISO 26262:2018, Road vehicles - Functional safety, Part 1: Vocabulary, Part 9: ASIL-oriented and safety-oriented analyses
ISO 21448:2022, Road vehicles - Safety of the intended functionality
Regulation (EU) 2019/2144 of the European Parliament and Council on type-approval requirements for motor vehicles, and their trailers, and systems, components and separate technical units intended for such vehicles, ... amending Regulation (EU) 2018/858 ...
SATW (2022): Autonomes Fahren. Ein Treiber zukünftiger Mobilität (in german). https://www.satw.ch/de/publikationen/publikation-zum-thema-autonomes-fahren
Yurtsever, E. et al, A survey of autonomous driving: Common practices and emerging technologies, IEEE Access, Vol. 8, 2020
A 1 Approaches to security requirements
The answer to the question "How safe is safe enough?" must not only be given from a technical perspective. A more comprehensive answer requires interdisciplinarity in order to include other important aspects and perspectives.
A 1.1 Approach from a technical perspective, regulatory-legal point of view
Requirements for the safety of automated vehicles (AV), underlying approaches and principles as well as associated validation procedures and methods have not yet been established but are in the process of being developed. Such requirements and validation procedures already exist for individual driving functions such as the emergency lane-keeping system [EU 2019/2144]. These include directives and regulations at UNECE and EU level, laws and ordinances at national level and relevant standards.
In its draft "Technical Guidelines" for automated driving systems (ADS), the Joint Research Centre of EC Ispra proposes9 with regard to their functional and operational safety:
- The manufacturer shall ensure freedom of unreasonable risks to vehicle occupants and other road users during the vehicle lifetime when compared with comparable transport services and situations within the Operational Design Domain (ODD).
- The manufacturer shall define acceptance criteria from which the validation targets are derived to evaluate the residual risk for the ODD taking into account accident data, data on performances from competently and carefully human driven vehicles and state-of-the-art technology.
Based on the general safety requirements, the need for quantitative target values/acceptability criteria is emphasised. Using current accident data for buses, cars, lorries and passenger cars in the EU, the EC Implementing Regulation [EC 2022/1426, Annex II, 7.1.1] considers an aggregated acceptance criterion of 10-7 fatalities per operating hour to be worth considering for the market launch of AVs in comparable services and situations.
An integrated safety management system is also required, which the manufacturer must implement, and which includes processes for collecting vehicle data.
The German ordinance regulating the operation of motor vehicles with automated and autonomous driving functions [BMDV 2022] was issued in anticipation of international requirements. It specifies the requirements of the law on autonomous driving10 and allows "vehicles without drivers" to participate in road traffic - initially, however, only in defined/authorised areas (ODD). It requires the manufacturer to develop a safety concept for functional safety, taking into account the state of the art:
9 EC JRC 2022, 1st JRC Workshop on Technical Guidelines, 13 July 2022
- The dangerous hazards, scenarios and events relevant to the ODD must be identified in a systematic procedure and assessed using a risk analysis.
- It must be demonstrated that hazards are recognised and reduced or avoided by means of suitable measures. In addition, according to the Road Traffic Act, a technical supervisor must be provided to return the vehicle from a risk-minimised state (after failure or overload of the ADS) to autonomous driving mode.
- Test cases must provide sufficient coverage for all scenarios, test parameters and environmental influences and be suitable for demonstrating that vehicles with autonomous driving functions are safer than vehicles driven by humans
- The state of the art is deemed to be met if the requirements of ISO standards 26262:2018 and 21448:2022 and UN Regulation No. 155 are fulfilled.
For highly or fully automated vehicles, French law stipulates that overall systems in which such vehicles are operated must guarantee at least the safety level of existing systems with comparable services or functions. If there are no comparable systems, the safety level must be determined by means of a safety study. The requirements for the safety of the transport system are also described using functional criteria11 .
The consultation draft of the Swiss ordinance on automated driving is aimed at SAE level 3 and 4 vehicles; level 5 is considered a "vision". The draft does not provide any information on vehicle safety and is primarily based on the European regulations in the context of "type approval". Switzerland will adopt these as overriding, transpose them into national law and should check their fulfilment in the course of vehicle approval (see [ASTRA 2023]). 12
Relevant standards, with different orientations, play a major role in the concretisation of security requirements, especially in the development phase of the AV:
- ISO 26262:2018, concerns the risk to functional safety following faults in electrical/electronic (E/E) systems or possible systematic faults such as undetected design and material faults, ageing, programming errors or inadequate updates. Proof is primarily provided in the development phase via so-called Automotive Safety Integrity Levels (ASIL): According to a predefined inductive method13 , possible faults in the system are assumed, their consequences (severity S), probability (probability E) and controllability (controllability C) are assessed and classes (letters with numbers) are assigned; finally, ASIL classes are formed by adding up the numbers after the letters, increasing from A to D (see Fig. A.1). For example, the ASIL class with the most severe injuries (S3), constant occurrence (E4) and difficult controllability (C3/D) indicates an assigned permissible probability of situational loss of function of 10-8 per hour.
- ISO 21448:2022, on the other hand, focuses on the safety of intended functionality (SOTIF) in a vehicle that is free from E/E faults but relies on the conscious perception of the driving
11 Art. R 3152-2 No. II and III Code des transports
12 The consultation was completed in February 2024; the ordinance is expected to enter into force in 2025.
13 Similar to the Failure Mode and Effects Analysis (FMEA). Safety analyses are also required to ensure that the risk of failing to meet safety targets due to the above-mentioned failures is low. Inductive (FMEA, HAZOP; ETA) and deductive (FTA) methods should be used, considering possible dependent failures; quantitative analyses require sufficient data [ISO 26262 - 9].
situation through complex sensors, algorithms and radio links. This standard assumes that functional safety has been demonstrated via ISO 26262 and focuses on the systematic identification of conditions and functional defects that could jeopardise the intended functionality of the vehicle; it recommends appropriate methods such as HAZOP and STPA (System Theoretic Process Analysis) and logical flow diagrams. SOTIF requirements are intended to ensure that unreasonable risks are not only mitigated/minimised but avoided altogether. These are risks that are considered unacceptable in the context of prevailing social and moral concepts.
Fig. A.1: Assessment matrix for determining ASIL classes - characterised by colours from green (covered by quality management QM) to red (serious)
For reasonable, acceptable residual risks, the standard requires a verification and validation strategy that shows that these are sufficiently small and fulfil quantitative requirements. These include a) the maximum number of accidents per operating hour/route or b) principles such as that the residual risk of a new system must not be greater than that of an existing reference system for the same purpose and area of use (GAMAB). Another admissible principle is the minimum endogenous mortality risk of a specific age and gender group (MEM), which must not be significantly increased by a new system14 . Cost-benefit considerations are permitted for a transitional area between the acceptable and unacceptable risk, in accordance with the principle of "as low as reasonably praticable" (ALARP), i.e. a further reduction of risks is not required if this would result in a disproportionate increase in costs.15
When deriving safety target values to avoid unacceptable risks, the relevant standards permit the use of values that are considered acceptable in other areas. Furthermore, the type of risk (voluntary, involuntary) and the exposure pathway (work) should be taken into account as influencing factors. The different principles can be traced back to the individual mortality risk and compared, see Fig. A.2.
14 MEM is a measure of the accepted (unavoidable) risk of death due to a technical, safety-relevant system, concretised in the EN 50126 standard as 2x10-4 deaths per person per year, the statistical risk of death of a European adolescent.
15 Values for both regions delivered, e.g., risks higher than participating in road traffic (about 10-4 per person, per year) are unacceptable, risks less than being struck by lightning (about 10-7 per person, per year) are widely accepted [R2A 2004].
UN Regulation No. 155 aims to identify inadequate defence and protection measures against cyber-attacks on the automated driving system, including communication channels, and No. 156 regulates software updates.
Fig. A.2: Comparison of acceptable and unacceptable mortality risks according to different principles and risk types (Source: U. Steininger, TÜV Süd, lecture 18-19 May 2022, Würzburg)
A 1.2 Consideration of ethical and social aspects
The opinion of the Ethics Commission of the then German Federal Minister of Transport and Infrastructure [BMVI 2017] provides indications of possible safety objectives from an ethical perspective, see also16 . According to this opinion, the protection of people takes precedence over all other utilitarian considerations, i.e. animals, the environment and property. The authorisation of automated systems is only justifiable if they promise at least a reduction in damage in terms of a positive risk balance compared to human driving performance under comparable conditions, taking into account all hazards and situations to which the new system is exposed. Technically unavoidable residual risks then do not prevent its introduction. Automated, networked technology should exclude or avoid the need for an abrupt handover of control to the driver and accidents as far as practically possible; critical situations, including dilemma situations (decision in favour of highervalue assets such as individual elderly persons versus groups of young people) should not arise in the first place; offsetting victims is prohibited. There is no ethical rule that always prioritises safety over individual freedom.
The EU implementing regulation requires that "in the event of an unavoidable alternative risk to human life, the ADS shall not provide any weighting on the basis of personal characteristics of humans" [EU 2022/2144. Annex II].
The existence of accidents in conventional road transport seems to be accepted as an inherent risk of the transport system.17
16 Armin Grunwald, Responsible design of autonomous driving. Ethical aspects and their relevance, in [SATW 2022]
17 Federal Highway Research Institute (BASt), Final report of the "Teleoperation research needs" working group, November 2023
It is generally expected that AVs will come to a standstill much more quickly than those controlled by humans and that ethical considerations will be incorporated into the programming guidelines for algorithms and that humans will retain or be able to retain control over them.
The relevant legislation on product liability and product safety provides an indication of which risks are socially acceptable and accepted today. The yardstick is the legitimate safety expectations of a reasonable user (third party) according to the specific circumstances for the intended or foreseeable improper use of the product. A product is defective if it does not correspond to the current state of science and technology at the time it is placed on the market. These principles also apply to automated vehicles, although there is still a need for legal clarification.
A 1.3 Needs of future customers
From an economic perspective - and based on experience with the slow introduction of sophisticated assistance systems - the question arises as to whether automated driving can satisfy relevant customer needs and thus be a business model at all. For business customers, this probably already applies to vehicles from SAE level 4 in certain areas and use cases in which considerable financial benefits can be achieved by reducing the number of drivers or even eliminating the need for a driver, such as for taxis, delivery services, provision on company premises and lorries in long-distance logistics transport. Based on the satisfaction with today's vehicles and their high level of safety, increased safety is a possible incentive for private customers to consume or buy, in addition to increased comfort and the possibility of participation in private transport by groups that were previously excluded (children, the elderly, people with limited mobility). This requires objectification and clear guidelines. It seems acceptable for the industry, as a key player, to develop and implement the safety requirements and provide evidence, while the authorities check their fulfilment and set guidelines.
The "affordable fulfilment of customer needs" has a significant influence on the social acceptance of AVs. Competitive and rivalry situations, differing expectations and the individual interests of companies and individuals can lead to differing risk assessments; particular interests can differ from social values and expectations. In addition, "humans" are usually very directly and "immediately" involved in the use of AVs (e.g. as vehicle owners, passengers, vulnerable road users), which can lead to additional risks in the event of malfunctions/behaviour: While in systems with high safety requirements, such as trains and aircraft, "experts" are usually able to monitor operation directly on site and quickly identify possible faults, this safety component is missing in AVs; it may have to be compensated for by other measures (control centre/technical assistance/remote operator, automated monitoring functions).
Acceptance (and thus the incentive to buy) can wane if the automated overall systems, including the communication channels to other vehicles (V2V) and the infrastructure (V2X), prove to be unreliable in operation, unexpected incidents accumulate, and the impression is created that humans are superior to the technical system and more trustworthy. Accordingly, reliable proof of reliability and the maturity of construction and operation are of foremost importance.
A 1.4 Proposed detection methodology
The principle of type or individual approval, which will probably continue to be pursued, contains the proof that the automated vehicle is sufficiently roadworthy; this is provided by the manufacturer and checked/tested by the authorities. EU directives and regulations are overriding and are adopted by the member and partner states by analogy and transposed into national law together with the requirements. The verification methodology proposed by the EU [EC 2022/1426] is based on general safety requirements and rests on three pillars:
1. Comprehensive documentation of the manufacturer (fulfilment of regulatory requirements, exclusion of unreasonable risks by means of analysis/simulation) and audit,
2. "track and real world testing",
3. Monitoring aftermarket authorisation with continuous additions to a scenario catalogue created in parallel scenario catalogue, in particular by comparing it with practical experience (accidents, near misses, unforeseen situations).
Given the lack of data for the new system, the usual statistical evidence cannot be provided (reliably). The predictive methods and non-specific (generic) data used are subject to considerable uncertainty, which decreases over time. This must be considered when defining safety requirements and could make their staggering over time (with possible tightening with increasing experience) appear advisable.
A 1.4.1 Proof of safety during the development phase
Tests are an essential element in the chain of safety verification through to authorisation. The obvious approach, primarily in the development phase, would be real-world testing. This would allow the sufficiently mature vehicle to be analysed as a complete system under real-world environmental and traffic conditions. The results obtained would be associated with a high degree of persuasiveness and trustworthiness. However, such a procedure is very costly and time-consuming18 and inefficient, also because previous tests could become invalid after improvements or updates and would have to be repeated. Furthermore, immature test vehicles could pose a danger to passengers and other road users.
Additional options include the installation of artificial faults and deliberate driving outside the intended operating range (ODD) as well as testing on test benches and in designated zones (bench & track testing). The relevant requirements have already been itemised.
In addition to widely practised physical testing, virtual procedures/replacement methods are necessary and postulated as permissible. These include:
- System models, usually with decomposition of the complex overall system into subsystems such as that for environmental perception, as well as models for drivers and driving tasks with decomposition of complicated driving tasks into subtasks.
18 According to RAND Corp. 2020, 440 million trouble-free kilometres would be required to demonstrate safety with high accuracy; with a fleet of 100 automated vehicles in continuous use at an average speed of 40 km/h, such a test procedure would take 12.5 years.
- analytical procedures for identifying risks, serious errors and hazards.
- Validated simulation techniques/tools to generate a variety of situational scenarios and identify "critical scenarios" for in-depth analyses and testing purposes.
As already explained, the semi-quantitative methodology similar to FMEA plays a key role in determining the ASIL classes and ultimately in verifying functional safety. It is widely used by manufacturers, accepted by authorities as the state of the art and randomly checked; the achievable quality/completeness depends on the effort involved, the data situation and the competence of the analysts. In addition, other well-known methods such as deductive fault tree analysis and inductive HAZOP studies, which work according to the decomposition and causality chain principle, are used with due scepticism. In addition, STPA is used, which models the overall system - and not independent subsystems.
In addition to combined analytical procedures, the individual proof of safety of intended functionality (SOTIF) is based on the evaluation of common accident statistics and data collected for AV, engineering judgement and the application of systematics such as tabular procedures and flow charts.
As already explained, the identification and analysis of assumed traffic situations (scenarios) is an essential element of safety verification. The automated driving system (ADS) is required to - recognises weaknesses and mistakes and reacts to them confidently,
- is in a "state of minimal risk" (e.g. by driving on the hard shoulder) if it leaves the intended area of use (ODD) or reaches its limits, - demonstrates anticipatory behaviour and interacts safely with other road users while observing traffic rules.
[EU 2022/1426, Annex III] requires "the consideration of the most relevant scenarios for the ODD" and distinguishes between two types of scenarios. For the scenarios listed in the "minimum set", the manufacturer must demonstrate, verified by the authority, that the "vehicle within its ODD" is free from unreasonable risks. These scenarios include maintaining and changing lanes, turning, crossing and emergency manoeuvres, avoiding collisions with other road users, safe driving on dual carriageways including merging and exiting manoeuvres, valet parking, etc.
The second type of scenario includes a large number of scenarios that are not included in the minimum set, but which constitute residual risks associated with the ADS. For these, it must be demonstrated that they can be considered tolerable in a social and moral context. A distinction is made between "nominal" (elements in the ODD), "critical" (special traffic and environmental conditions, communication breakdowns, etc.) and "failure scenarios". Principles are proposed for their identification; a distinction is made between "data-based" and "knowledge-based" methodological approaches. There are only general guidelines (inductive methods) and individual specific proposals (STPA) for the analysis methods to be used; requirements such as the presentation of the methodological approach and assumptions as well as the limits and uncertainties are outlined for the virtual toolchains used. The manufacturer is ultimately responsible for the selection of suitable methods and the quality of the verifications; he is granted a large degree of discretion, albeit taking into account the state of the art.
A 1.4.2 Proof of safety during the operating phase
As statistical proof that the highly automated driving system/vehicle fulfils all safety requirements cannot be reliably provided before market approval, requirements regarding monitoring and surveillance are emerging for the operating phase [2022/1426, Annex II]. It is expected that vehicles will have to be equipped with an integrated memory (black box) to record event-based data. Manufacturers are to be obliged to record serious malfunctions and accident events as well as activations and deactivations of the ADS and use them for ongoing software updates and system improvements; safety-critical incidents are to be reported immediately to an official body designated for this purpose. Furthermore, real-life scenarios and critical situations must be compared with those that have been analysed in advance and on which the approval was based.
Safety is therefore not a constant factor and should be able to increase continuously but may even decrease at times due to faulty improvements.
In the event that the ADS exhibits faults, is overloaded or reaches its limits, measures must be provided and validated to bring the vehicle into a "minimum risk" state and return it to operating condition with or without external assistance19
A 1.4.3 Assessment of the performance of required detection methods
There are established standards and frameworks, some with flowcharts, which deal with the functional and operational safety of automated driving systems (ADS). The recommended methods mainly follow the principle of breaking down the overall system and the driving tasks into individual elements and the approach of linear causal chains without feedback.
EC JRC 2022 considers it necessary and feasible to define an acceptability criterion, e.g. based on fatality rates per operating hour or distance travelled. Detection methods for determining the probabilities of situations (conditions) that could lead to the death of the driver (or vehicle occupants) and/or other road users are considered "developable" by combining existing methods and approaches prior to market launch; suitable steps for determining dominant scenarios are recommended.
From the point of view of safety requirements and detection methods, a distinction should be made between the initial authorisation phase and the market penetration phase, with initially very limited empirical and statistical data combined with considerable uncertainties and the "potential for surprises" remaining at the end of the development phase, and then a large data pool, increased field observations and thus more reliable evidence.
A 2 Principles and derived safety targets for Switzerland in other sectors
In general, protection target concepts based on probabilistic risk analysis and risk assessment with quantitative targets have a certain tradition in Switzerland, similar to the Netherlands and the UK.
19 The German regulation provides for an "event-based", highly qualified person appointed as a technical supervisor (TA) who, among other things, should enable and support the exit from the minimum risk state; the Swiss draft regulation proposes an "operator".
In contrast, other countries such as Germany (partly due to a different legal system) tend to pursue deterministic concepts20 .
A 2.1 Protection of the population and the environment from serious damage caused by incidents
The Hazardous Incident Ordinance (StFV)21 covers risks emanating from plants with a chemical or biological hazard potential and from the transport of dangerous goods or their transport via pipelines. Third parties outside and inside the site are considered when determining the damage.
The risk is presented as a diagram, formed from the cumulative probability of occurrence per year (abscissa) and the extent (ordinate) of a large number of incident scenarios. In addition to fatalities (manifested within 30 days) and injuries, other types of damage are recorded and combined to form an incident value. Lines demarcate the range of acceptable and unacceptable risks, with a transitional area in between.
A tenth of the average individual probability of death (10-3 per year) serves as an anchor point to the unacceptable range, the difference to the acceptable range is two orders of magnitude. Damage is weighted more heavily using an aversion coefficient with a (politically set) value of 2 The incident value of 0.6 is assigned a number of 100 fatalities, with a corresponding probability per year and plant of 10-7 and 10-9. These values would be most suitable for use as a benchmark.
A 2.2 Protecting the public from nuclear power plant accidents
Target values in the field of nuclear technology do not exist in the direct form of admissible risks, but via auxiliary values ("target values") as authorisation requirements for safety installations. Traditionally, the focus has been on the fulfilment of deterministic requirements, i.e. proof of control of the worst-case accident, against which the plant is to be designed without serious environmental impacts being feared. In addition, it must now be demonstrated that the total occurrence frequency of core meltdown accidents (core damage frequency, CDF) is below a certain value, e.g. 104 or 10-5 per reactor year for old and new plants respectively. For core meltdown accidents combined with early failure of the containment and release of large quantities of radioactive substances into the environment, a value an order of magnitude lower generally applies. The proofs are provided using the methodology of probabilistic risk analyses (PRA), the results are periodically updated; they are also reflected in operating and accident experience with operating plants, which have a cumulative wealth of experience of around 18,000 reactor operating years worldwide. The consequences of severe accidents that exceed the design limits of the plant (maximum credible accidents) for the surrounding area have been assessed as part of academic studies. The number of immediate and delayed radiation-related fatalities and the number of people affected by protective measures as well as the amount of financial damage are used as measures of damage; psychological damage is usually not recorded. The results are used to estimate societal nuclear accident risks and categorise them in the risk landscape.
20 Det Norske Veritas 2010
21 The StFV came into force in April 1991 in response to the fire disaster in Schweizerhalle in November 1986, is periodically adapted to new circumstances and supplemented by a modular manual [FOEN 2018].
A 2.3 Rail transport
The rail traffic within the railway network is not (yet) automated, but there are digitalised assistance systems as well as control and safety systems. The assessment of safety is part of a process package (RAMS), which also includes reliability, availability and maintenance and is specified in a standard [EN 50126]. Proof of safety is understood as a process (see Fig. A.3).
Fig. A.3: Safety process according to EN 50126/2 statements on safety, i.e. the absence of unacceptable systemic risks:
- Statistical values based on operating experience (risk analysis, safety figures).
- Information on the process flow (compliance with regulations, risk management procedures in accordance with Regulation (EU) 402/2013, company culture).
- Fulfilment of acceptability criteria (risk acceptance categories, see Fig. A.4, SIL22, MEM, ALARP23, GAMAB/GAME).
The mortality limit (deaths per person, per year) for rail travellers is set at ≤ 8.8x10-5, the effective value24 for Swiss railways is ≈ 5x10.-8
22 For applications of the EN 50126 standard, generally SIL4 with fault tolerance < 10-9 (excl. fail-safe).
23 "reasonably achievable": An amount of CHF 6.5 million is to be applied as the safety benefit for one prevented fatality for each of the groups "residents", "travellers on the train" and "employees" (Federal Office of Transport (FOT) safety policy).
24 According to the method for assessing the individual risk of the FOT.
A 2.4 Air transport
The automation of aviation and aircraft control is increasing, but is only comparable to automation level SAE-2 because flight monitoring and control is the responsibility of the crew. Accidents are rare and usually occur during take-off or landing. Accordingly, accident rates are given per flight (and not per distance travelled). Typical values are one fatal accident per 10 million flights and seem to be accepted as "involuntary" risk exposure.
Data analyses show that the introduction of a new type of technology increases the number of fatal accidents, which then decreases and 5 to 10 years after the introduction phase falls below the value of the machines previously used, the "old" state of the art [Junietz et al. 2019].
The Federal Office of Civil Aviation (FOCA) has drawn up a risk matrix for dealing with risks in Swiss aviation (primarily airport-related, including all specialist areas, from the perspective of aircraft occupants), with the expected extent of damage and associated frequency of occurrence as axes25 and a transition area (yellow) between the acceptable (green) and unacceptable (red) risk, in which the ALARP principle may be applied. The boundary lines take risk aversion into account; for severity class M (2-6 fatalities), for example, the acceptable probability is max. 10-7, the unacceptable 10-5 , in each case per movement on the ground/flight hour (see Fig. A.5).
25 Based on the methodology and risk matrix with transitional area of Aviation Risk Management Solutions (ARMS); the European Aviation Safety Agency (EASA) issued the European Risk Classification Scheme (ECRS) in 2022 to classify individual events, but it does not contain any statements on risks and their admissibility.
A 2.5 Socially accepted emergency risk
The recently published risk analysis "Disasters and Emergencies in Switzerland 2020" [Federal Office for Civil Protection (FOCP), 17.11.2021] provides an indication of the de facto socially accepted risk in Switzerland. It quantifies aggregated damage, such as that caused by a serious nuclear power plant accident, at several tens of billions of Swiss francs with a frequency of occurrence of once every three million years; the consequences of an aircraft crash are estimated at several billion Swiss francs with a frequency of once every 30 years, while the consequences of a hazardous goods accident involving railways are estimated at just over one billion Swiss francs with a frequency of once every three million years. The highest societal risk is seen in an electricity shortage, with damage totalling several hundred billion francs at a frequency of once in 30 years (see Fig. A.6).
A.6: Risk diagram for non-intentional events
About the SATW
The Swiss Academy of Engineering Sciences SATW is the most important network of experts in the field of engineering sciences in Switzerland and is in contact with the highest Swiss bodies for science, politics and industry. The network consists of elected individual members, member organisations and experts.
On behalf of the federal government, SATW identifies industrially relevant technological developments and informs politicians and society about their significance and consequences. As a unique specialist organisation with a high level of credibility, it provides independent, objective and comprehensive information on technology – as a basis for forming well-founded opinions. The SATW also promotes interest in and understanding of technology among the general public, especially young people. It is politically independent and non-commercial.