Submitted by the expert from France

Informal document GRVA-18-34 18th GRVA, 22–26 January 2024 Provisional agenda item 5 (a)

List of relevant GRVA Regulations for software updates

This document was prepared by the expert from France. It proposes a list of the relevant UN Regulations under the purview of GRVA, for the purposes of software updates.

I. List of relevant UN Regulations UN Regulation Title Relevant (Yes/No) UN Regulation No. 13 Braking Yes UN Regulation No. 13-H Braking (passenger cars) Yes UN Regulation No. 78 Braking (Category L) Yes* UN Regulation No. 79 Steering equipment Yes UN Regulation No. 89 Speed Limitation Devices Yes UN Regulation No. 90 Replacement brake lining

assemblies, drum brake linings and discs and drums for power-driven vehicles and their trailers

No

UN Regulation No. 130 Lane Departure Warning System Yes UN Regulation No. 131 Advanced Emergency Braking

System Yes

UN Regulation No. 139 Brake Assist System Yes UN Regulation No. 140 Electronic Stability Control Yes UN Regulation No. 152 Advanced Emergency Braking

System Yes

UN Regulation No. 155 Cyber security Yes UN Regulation No. 156 Software update and software update

management system Yes

UN Regulation No. 157 Automated Lane Keeping System Yes

II. Justification

During the 191st session of WP.29, the expert from France proposed that all subsidiary Working Parties of WP.29 consider the list of the Regulations under their respective purview which may be relevant for the purposes of software updates (see ECE/TRANS/WP.29/1175, para. 37). This identification of relevant Regulations is necessary to introduce the necessary provisions on identification and update of software in each affected Regulation, using the guidance provided in Annex 7 of the Consolidated Resolution on the Construction of Vehicles (R.E.3).

This document proposed a list of the UN Regulations under the purview of GRVA which may be relevant for the purposes of software updates. More specifically:

- UN Rs13, 13-H, 78*, 79, 130, 131, 139, 140, 152, 157 (Regulations on braking, steering, active safety and ADAS/ADS): Most systems directly or indirectly related to braking and steering greatly rely on software as part of their basic operation.

- UN R89 (speed limitation devices and functions): Most speed limitation devices and functions greatly rely on software as part of their basic operation.

- UN R90 (replacement brake parts): This Regulation only applies to physical replacement parts. If no software is present on these replacement parts, this Regulation is not relevant for the purposes of software updates at this stage. However, any introduction of connected replacement parts on the market should lead to a reconsideration of this classification.

2

- UN R155: Exact software versions of critical electronic control units (such as ECUs with external interfaces) are fully relevant for the purposes of type approval pursuant to R155.

- UN R156: Software versions of relevant electronic control units (such as telematics, infotainment or human-machine interfaces) may have an impact on the definition of a vehicle type with regard to R156.

Additionally, the authors believe that Global Technical Regulations No. 3 and 8 are also relevant for the purposes of software updates. However, they are outside of the scope of the Regulations for which guidance is given by Annex 7 of R.E.3. * UN R78 is a Regulation that only applies to vehicles of Category L, which are not part of the scope of UN R156. GRVA may wish to consider whether an extension of the scope of UN R156 is necessary before including provisions for software updates within UN R78.

Submitted by the experts of France (leader of the workshops group on UN-R 152) on behalf of the workshops group on UN-R 152

Informal document GRVA-18-24 18th GRVA, 22-26 January 2024 Provisional agenda item 7

Proposal for a supplement to 01 series of amendments to UN Regulation No. 152 (Advanced Emergency Braking System for M1 and N1 vehicles)

The text produced below was prepared by the workshops group on UNR-R 152 and replaces the working document GRVA/2023/22. The proposal is aimed at allowing the applicant to use virtual testing methodology as alternative methodology to the physical tests.

I. Proposal

Add a new paragraph 2.18., amend to read:

“2.18. “Virtual testing” is the process of testing a system using one or more simulation models.”

Add a new paragraph 6.7., amend to read:

“6.7. Virtual testing of dynamic tests

6.7.1. Virtual testing may be used by request of the vehicle manufacturer as an alternative for the tests described in paragraphs 6.4. to 6.6.,. The provided virtual testing shall be verified and validated according to and are used in accordance with annex 4.

6.7.2. Virtual testing may be used in the evaluation of the warning and activation tests in accordance with paragraph 1.8. of Schedule 3 and Schedule 8 of Revision 3 of the 1958 Agreement.

[6.7.3. In addition to the simulation results, simulated test runs [shall / may] be conducted as physical tests as well on the request of the type approval authority and technical service.]

6.7.4. In case of virtual testing is chosen by the manufacturer, a separated report including at least the additional data information specified in annex 4 paragraph 1.5. shall be annexed to the test report.”

GRVA-18-24

2

Add a new Annex 4, to read:

“Annex 4 - Virtual testing of dynamic tests

Introduction (for information only)

This annex describes the method that can be used to consider virtual testing as an alternative to physical testing, based on the manufacturer request.

This method is mainly based on 2 separate pillars:

- Pillar 1 : the validation of the virtual testing method by comparison with physical results and,

- Pillar 2 : the virtual testing results for approval process.

1. Validation of the virtual testing method (pillar 1)

1.1. General specifications

1.1.1. The manufacturer shall provide documentation to prove the credibility of the virtual testing results.

1.1.2. The vehicle manufacturer shall define the validity domain on which the virtual testing will be applicable. This annex only applies within this validity domain.

1.1.3. Credibility of the virtual toolchain that is used for the virtual testing shall be demonstrated by the vehicle manufacturer to the Type Approval Authority and Technical Service.

For this, the following five criteria shall be considered :

(a) Capability – what virtual toolchain can do, and what are the associated risks;

(b) Accuracy – how well virtual toolchain does reproduce the target data;

(c) Correctness – how sound & robust is the used data and the algorithm in the tools;

(d) Fit for Purpose – how suitable is the virtual toolchain for the assessment (e.g. vehicle dynamic model, sensor model, system control model, environment model, scenario model, targets model, …) within its validity domain.

(e) Usability – What training and experience is needed and what is the quality of the process that manage its use.

1.2. Physical validation tests

1.2.1. At the request of the technical service, in addition to the documentation provided by the vehicle manufacturer, physical tests shall be performed or witnessed to confirm the accuracy between the physical and the simulation results.

1.2.1.1 The number of physical tests to be tested shall be defined in agreement between the manufacturer and the technical service in order to sufficiently cover the validity domain specified by the vehicle manufacturer.

GRVA-18-24

3

1.2.2. The number of tests performed shall ensure a statistical comparison between physical and simulation results.

1.3. Simulation model

1.3.1. The simulations (including development of the model) shall be run under the responsibility of the vehicle manufacturer. It shall reflect the architecture of the vehicle, system and components to be tested in relation to the requirements of the current regulation on the specified validity domain.

1.3.2. The models that are developed and tested shall be capable of accurately representing the relevant aspects of the physical AEBS system that is being modelled. The models are used in tools and the tools are incorporated into toolchains which emulate the overall physical behaviour of the AEBS system with the appropriate quality within the declared domain of validity.

1.4. Simulation model validation process

1.4.1. The simulation model shall be validated in comparison with the physical validation tests performed under paragraph 1.2. and comparability of the test results shall be proven.

1.4.2. The validation strategy shall be based on scientific methods, defined by the car manufacturer and agreed with the type approval authority and technical service.

1.4.3. For the validation, key performance indicators shall be assessed such as time to collision, remaining distance or impact speed.

1.5. Additional data and information

For this application, the following information shall be supplied to the approval authority and technical service in addition to the data, and drawings listed in paragraph 3.2. of this Regulation.

1.5.1. A description of the applied simulation and calculation method which has been used such as identification of the model, the analysis software, its producer, its commercial name, the version and contact details of the developer.

1.5.2. A description of the input parameters.

1.5.3. A description of the validity domain taking into account AEBS performance influencing factors.

1.5.4. All parts of the simulation toolchain such as interlinked simulation modules and tools shall be described by the manufacturer.

1.5.5. The methodology used to generate physical validation data, such as data recording equipment, data processing, calculation of scalar valuesshall be documented in the simulation report.

1.5.6. A description of the data management archiving system shall be provided by the manufacturer.

1.5.7. A description of the versions control and the review processes in case of modification within the simulation toolchain shall be provided by the manufacturer.

GRVA-18-24

4

2. Virtual testing results for approval process (pillar 2)

2.1. Compliance of the Advanced Emergency Braking System with the performance requirements as defined in Paragraphs 5.2.1 to 5.2.3 of this regulation may be demonstrated by the vehicle manufacturer to the Type Approval Authority or Technical Service by making use of virtual testing of the dynamic maneuvers of the paragraph(s) 6.5 to 6.7 of this Regulation.

2.2. All simulation results provided by the manufacturer in applying for an approval in accordance with paragraph 4. of this regulation shall refer to the method evaluated and validated according to paragraph 1 of this annex.

2.3. Additional data and information

For this application, the following information shall be supplied to the technical service in addition to the data, and drawings listed in paragraph 3.2. of this Regulation.

2.3.1. A description of the applied simulation method which has been used such as identification of model, the analysis software, its producer, its commercial name, the version and contact details of the developer.

2.3.2. A description of the input parameters.

2.3.3. A reference to the validated simulation method used in application of paragraph 1 of the current annex.

2.3.4. All parts of the simulation toolchain such as interlinked simulation modules and tools shall be described by the manufacturer.”

II. Justification

1. This proposal targets to let the opportunity to applicant to use virtual testing methodology as alternative methodology to the physical tests. As it is already defined at European Union Whole Vehicle Type Approval system (WVTA), in other regulations or in the current activities on automated driving systems by the Informal Working Group on Validation Method for Automated Driving (VMAD) Subgroup 2, this approach requires the preliminary assessment of the methodology to be used.

2. This proposal defines a practical approach to preserve safety main principles letting the flexibility to the applicant in the virtual tools to be used.

3. An example of the application is presented in informal document GRVA-15-20.

Note by the secretariat: this amendment proposal, if adopted as supplement to the 02 series of amendments, would require adjustments as para. 6.7. already exist in the 02 series of amendments, reading:

“6.7. Warning and Activation Test with a Bicycle Target”

Submitted by the experts of France (leader of the workshops group on UN-R 152) on behalf of the workshops group on UN-R 152

Informal document GRVA-18-23 18th GRVA, 22-26 January 2024 Provisional agenda item 7

Confidential C

Proposal for a supplement to 01 series of amendments to UN Regulation No. 152 (Advanced Emergency Braking System for M1 and N1 vehicles)

The text produced below was prepared by the workshops group on UNR-R 152 and is based on working document GRVA/2023/22. The proposal is aimed at allowing the applicant to use virtual testing methodology as alternative methodology to the physical tests. The modifications to the existing text GRVA/2023/22 are marked in bold red for new or strikethrough for deleted characters.

I. Proposal

Add a new paragraph 2.18., amend to read:

“2.18. “Virtual testing” is the process of testing a system using one or more simulation models.

Add a new paragraph 6.7., amend to read:

“6.7. Computer simulation Virtual testing of dynamic tests

6.7.1. A computer simulation model Virtual testing may be used by request of the vehicle manufacturer as an alternative for the tests described in paragraphs 6.4. to 6.6.,. The provided virtual testing the simulation model and simulation toolchain have been shall be verified and validated according to and are used in accordance with annex 4.

6.7.2. Simulation tools and mathematical models Virtual testing may be used in the for evaluation of the warning and activation tests may be used in accordance with paragraph 1.8. of Schedule 3 and Schedule 8 of Revision 3 of the 1958 Agreement. Manufacturers shall demonstrate the scope of the simulation tool, its validity for the scenario and concrete vehicle concerned as well as the validation performed for the simulation tool chain (correlation of the outcome with physical tests) in accordance with annex 4.

6.7.3. The technical service shall be able to validate the simulation model using physical validation tests.

[6.7.3. In addition to the simulation results, at least 30% of the simulated test runs [shall / may] be conducted as physical tests as well on the request of the type approval authority and technical service. The results of simulated test runs and physical test runs shall be checked for differences on an individual basis and using standard statistical tests by the technical service.]

GRVA-18-XX

2 Confidential C

6.7.4. In case the computer simulation of dynamic tests of virtual testing is chosen by the manufacturer, a separated report including at least the additional data information specified in annex 4 paragraph 1.5. shall be annexed to the test report.”

Add a new Annex 4, to read:

“Annex 4

Computer simulation Virtual testing of dynamic tests

Introduction (for information only)

This annex describes the method that can be used to consider virtual testing as an alternative to physical testing, based on the manufacturer request.

This method is mainly based on 2 separate pillars:

- Pillar 1 : the validation of the virtual testing method by comparison with physical results and,

- Pillar 2 : the virtual testing results for approval process.

This annex describes the processes that can be used to consider simulation results instead of physical results demonstrating compliance with regulatory requirements.

These processes allow both to optimise the reactivity of manufacturers to cover different vehicle definitions and to optimise the economic aspect by limiting the number of physical means involved.

However, this approach is only possible if the framework of the process is clearly defined and if the level of confidence in the results presented is sufficient and based on objective criteria of physical representativeness.

This approach is mainly based on 2 separate axes: the validation of the simulation method and the simulation results for approval process.

The validation of the simulation method is a key stage in the comprehensive digital validation process because it defines the mathematical model’s level of representativeness with respect to the physical test. The quality of the correlation is therefore critical and is assessed via a simulation / test comparison. Once the model has been correlated or, in other words, when the behaviour calculated is similar to the behaviour of the subject in the actual tests, the model can be used to predict the subject’s behaviour within its validity domain.

The simulation results for approval process are the final stage of the whole procedure, namely the type-approval of a vehicle in respect of a regulatory act based solely on a virtual type-approval. Once the digital model’s representativeness has been demonstrated within a scope of validity, this process can be used to assess the performance of the model tested against the requirements of the regulatory text.

This global approach is summarized step by step in the scheme below figure 5 and further detailed in the following chapters.

Figure 5 Generic flow chart of the “Computer simulation of dynamic tests as an equivalent approval method”

GRVA-18-XX

3 Confidential C

1. Validation of the simulation virtual testing method (pillar 1)

In order to guarantee that the simulation method used by the manufacturer is able to provide representative results acceptable for approval process, this simulation method shall be evaluated and validated by the technical service.

1.1. Definition of the validity domain General specifications

1.1.1. The car manufacturer shall define the boundary conditions for the simulation method. These boundary conditions define the limits within the simulation method can be used. The manufacturer shall provide documentation to prove the credibility of the virtual testing results.

1.1.2. The validity domain definition shall cover both vehicle characteristics (e.g. mass, equipment, exact sensor type, control algorithm) and scenario characteristics (e.g. speeds, target). The vehicle manufacturer shall define the validity domain on which the virtual testing will be applicable. This annex only applies within this validity domain.

1.1.3. Depending on the validity domain required by the manufacturer, the Technical Service will define the matrix of vehicle and scenarios to be tested in order to cover the entire domain, in accordance with paragraph 1.2. Credibility of the virtual toolchain that is used for the virtual testing shall be demonstrated by the vehicle manufacturer to the Type Approval Authority and Technical Service.

GRVA-18-XX

4 Confidential C

For this, the following five criteria shall be considered :

(a) Capability – what virtual toolchain can do, and what are the associated risks;

(b) Accuracy – how well virtual toolchain does reproduce the target data;

(c) Correctness – how sound & robust is the used data and the algorithm in the tools;

(d) Fit for Purpose – how suitable is the virtual toolchain for the assessment (e.g. vehicle dynamic model, sensor model, system control model, environment model, scenario model, targets model, …) within its validity domain.

(e) Usability – What training and experience is needed and what is the quality of the process that manage its use.

1.2. Physical validation tests

1.2.1. At the request of the technical service, in addition to the documentation provided by the vehicle manufacturer, The technical service, physical tests shall be performed or witnessed tests to prove confirm the validity accuracy between the physical and of the simulation model results.

1.2.2. 1.2.1.1 The number of scenarios physical tests to be tested shall be defined in agreement between the manufacturer and by the technical service in order to sufficiently cover the validity area domain requested specified by the vehicle manufacturer.

1.2.3. At least 10 repetitions of worst cases scenarios shall be performed and results of the stop relative distance from target or target impact velocity shall be inside a defined interval from the median value. This interval is defined by the technical service.

1.2.3.1. Worst case scenarios are those where model uncertainties are expected to have the greatest impact on the representativeness of the simulation model (e.g. impact with target during full braking would lead to a significant spread in results, lowest possible speed for car-bicycle-scenarios where sensor angle is most relevant).

1.2.4. As mentioned under paragraphs 6.10. of this regulation on the robustness of the system, some physical tests may be repeated in case the system fails to meet the performance requirements. The number of repeated tests shall not exceed:

(a) 10.0 per cent of the performed test runs for the Car to Car tests; and

(b) 10.0 per cent of the performed test runs for the Car to Pedestrian tests.; and

(c) 20.0 per cent of the performed test runs for the Car to Bicycle tests.

1.2.52. The physical tests used for building a physical reference for the numerical model validation shall be repeatable. The repeatability shall be evaluated on the impact speed or remaining distance values of the 10 repetitions which shall be within a corridor defined by the technical service around the median value of the physical tests. The number of tests performed shall ensure a statistical comparison between physical and simulation results.

GRVA-18-XX

5 Confidential C

1.3. Simulation model

1.3.1. The simulations (including development of the model) shall be conducted run under the responsibility of by the vehicle manufacturer. It shall reflect the complexity of the architecture of the vehicle, system and components to be tested in relation to the requirements of the current regulation and its boundary conditions on the specified validity domain.

1.3.2. The model shall be capable of describing the real physical behaviour on the validity domain. 1.3.3. The simulation model shall be constructed, and assumptions prescribed, in such a way that the calculation gives conservative solution, in which the result is independent of the incremental time step.

The models that are developed and tested shall be capable of accurately representing the relevant aspects of the physical AEBS system that is being modelled. The models are used in tools and the tools are incorporated into toolchains which emulate the overall physical behaviour of the AEBS system with the appropriate quality within the declared domain of validity.

1.3.4. In addition to the parameters listed in paragraph 1.4. of the current annex, at least the following elements have to be defined in the simulation model:

(a) Vehicle dynamic model including transmission, power train, etc;

(b) Sensor model;

(c) ADAS control model;

(d) Environment model;

(e) Scenario model;

(f) Target model for pedestrians, cyclists and cars;

The technical service shall check the model for correct physical behaviour.

1.4. Simulation model validation process

1.4.1. The simulation model shall be validated in comparison with the physical validation tests performed under paragraph 1.2. and comparability of the test results shall be proven.

1.4.2. The model shall be checked against the repeatability tests and the median value defined as specified in paragraph 1.2.5.

The validation strategy shall be based on scientific methods, defined by the car manufacturer and agreed with the type approval authority and technical service.

1.4.3. The simulation model shall be considered valid in the requested validity domain if, based on a significance level of 5%, there is no reason to believe that the simulation model results and the test results come from two different distributions for at least the following key performance indicators:

GRVA-18-XX

6 Confidential C

(a) Time to collision FCW in s;

(b) Mean vehicle speed between 4s TTC and before AEB activation in km/h (= initial speed);

(c) Average of absolute Lateral deviation in m;

(d) Brake distance in m (only for test cases with avoidance);

(e) Mean fully developed brake deceleration in m/s²;

(f) Remaining distance to the target after standstill in m (set to zero for non-avoidance);

(g) Impact speed into target in km/h set to zero for avoidance);

(h) Brake force build-up time from start of braking to maximum brake deceleration in s;

(i) Time to collision for start of braking in s (align wording with Euro NCAP test procedure).

Standard significance tests shall be used by the manufacturer.

For the validation, key performance indicators shall be assessed such as time to collision, remaining distance or impact speed.

1.4.4. It shall be verified that the measured data describes the correct physical quantities. This means it needs to be checked for plausibility and filtered appropriately. If quantities are not measured directly, an argumentation is required to show that they still can be used.

1.5. Additional data and information

For this application, the following information shall be supplied to the approval authority and technical service in addition to the data, and drawings listed in paragraph 3.2. of this Regulation.

1.5.1. A description of the applied simulation and calculation method which has been used such as with identification of the model, the analysis software, including at least, its producer, its commercial name, the version and contact details of the developer.

1.5.2. A description of the input parameters encoding the models used including at least systems functionalities characterization, mechanical hypothesis, values for defined masses, centre of gravity, moments of inertia and boundary conditions.

1.5.3. A definition description of the validity domain taking into account based AEBS performance influencing factors. vehicle parameters as mass distribution, speed ranges, etc. used in the application of paragraph 1.1. of the current annex.

1.5.4. Each of the All parts of the simulation toolchain calculation such as interlinked simulation modules and tools shall be described detailed by the manufacturer : pre-processing, processing and post-processing including a justification of the normal termination of the simulation (post processing logfile for example).

1.5.5. The methodology used to generate physical validation data test correlated data, such as (at least but not limited to: data recording equipment, data processing, calculation of scalar values, statistical calculations,

GRVA-18-XX

7 Confidential C

performance indicator values as specified in paragraph 1.4.3., results of the statistical calculations ) shall be documented in the simulation report.

1.5.6. A description of the data management archiving system and the updates management process (braking system design, soft updates, regulation amendments) shall be provided by the manufacturer.

1.5.7. A description of the versions control and the review processes in case of modification within the simulation toolchain shall be provided by the manufacturer.

2. Simulation Virtual testing results for approval process (pillar 2)

2.1. The manufacturer may provide simulation results to meet the requirements specified in paragraphs 6.4. to 6.6. of this Regulation only if the method used to obtain the results have already been evaluated and validated in application of the current annex. Compliance of the Advanced Emergency Braking System with the performance requirements as defined in Paragraphs 5.2.1 to 5.2.3 of this regulation may be demonstrated by the vehicle manufacturer to the Type Approval Authority or Technical Service by making use of virtual testing of the dynamic maneuvers of the paragraph(s) 6.5 to 6.7 of this Regulation.

2.2. All simulation results provided by the manufacturer in applyingication for of the an approval following in accordance with paragraph 4. of the current this regulation shall refer to the method previously evaluated and validated according to paragraph 1 of this annex. in application of the current annex.

2.4. Additional data and information

For this application, the following information shall be supplied to the technical service in addition to the data, and drawings listed in paragraph 3.2. of this Regulation.

2.4.1. A description of the applied simulation and calculation method which has been used such as with identification of model, the analysis software, including at least, its producer, its commercial name, the version and contact details of the developer.

2.4.2. A description of the input parameters encoding the models used including at least systems functionalities characterization, mechanical hypothesis, values for defined masses, centre of gravity, moments of inertia and boundary conditions.

2.4.3. A reference to the validated simulation method used in application of paragraph 1 of the current annex.

2.4.4. Each step of the calculation All parts of the simulation toolchain such as interlinked simulation modules and tools shall be described detailed by the manufacturer: pre-processing, processing and post-processing including a justification of the normal termination of the simulation.”

II. Justification

1. This proposal targets to let the opportunity to applicant to use virtual testing methodology as alternative methodology to the physical tests. As it is already defined at

GRVA-18-XX

8 Confidential C

European Union Whole Vehicle Type Approval system (WVTA), in other regulations or in the current activities on automated driving systems by the Informal Working Group on Validation Method for Automated Driving (VMAD) Subgroup 2, this approach requires the preliminary assessment of the methodology to be used.

2. This proposal defines a practical approach to preserve safety main principles letting the flexibility to the applicant in the virtual tools to be used.

3. An example of the application is presented in informal document GRVA-15-20.

Note by the secretariat: this amendment proposal, if adopted as supplement to the 02 series of amendments, would require adjustments as para. 6.7. already exist in the 02 series of amendments, reading:

“6.7. Warning and Activation Test with a Bicycle Target”

List of tables

Table1

Table2

Table 2a

Table 2b

Table 3

Table 4

Table 5

Table 5a

Table 6

Table 6a