DGITM/DMR/TUD GRVA-Scenarios-02
6-7 May 2024
Proposal for a workshop on scenarios under UNECE fora
Preparation of the 1st meeting, Paris La Défense, May 6-7 2024
This document intends to shape questions underlying the initiative of a dedicated UNECE workshop on
scenarios and their database’s management. In preparation of 18th GRVA, ITU shared interest to further
discuss “scenario catalogues for ADS”.
During last GRVA, several presentations were made to elaborate on the needs to work on scenarios according
to different perspectives. ITU presented interest in working on a scenario catalogue and shared strong
support from contracting parties and industry in creating a scenario catalogue.
France and the United Kingdom shared the need to foster collaboration on scenarios; especially on the value
harmonization could bring to the entire ecosystem, for both public and private stakeholders. ADScene and
SafetyPool, either two existing database holders presented two different ways to implement a national
scenario database, from an industrial or a research perspective.
Based on these presentations, the opportunity of a new sub-group under UNECE fora came as a possible way
forward.
This raises questions such as the objectives of this group and its possible production, common addressable
topics on scenarios that need further discussions. In this context, ahead of the meeting scheduled in Paris on
6 and 7 May 2024, this document intends to collect participants’ views on these questions.
The document is organized as follows: it tries to shape the objectives of the work to achieve around scenario
issues; it reviews briefly the current state of international work on scenarios, and finally opens up questions
on opportunities and challenges to be discussed during the workshop (how to handle each). In that manner,
this document identifies potential gaps towards a better use of scenarios in proposing a draft agenda.
1- Objectives
Scenarios are at the forefront of safety demonstration within the international ADS framework, in which are
integrated safety assessment and test methods: as one of the five NATM pillars and further by making the
link between these pillars, having the advantage of being independent of test procedures themselves.
Scenarios have the potential to address the complexity of assessing ADS performances (while remaining
technology neutral) across the diversity of possible ODDs (and more locally ODs).
During last GRVA meeting, discussions pointed out the need to better assess international coordination needs
before examining the suitability of a centralized scenario catalogue or database.
Therefore, it is proposed that the terms “UN scenario catalogue” would be understood more broadly than a
physical platform. In this perspective, the scope of the group would rather be: “International exchanges,
cooperation or coordination in using scenario approaches, catalogues or databases for safety
demonstration of automated road transport systems”.
Working on a cooperative or coordinated approach would address several challenges of the scenario-based
approach for systems’ validation:
- How to tend towards a better scenario coverage, as inherent to the notion of “reasonably foreseeable
hazards within the ODD”? : it is likely that diverse scenario approaches, catalogues and database could
bring a significant value-added one to another, in helping filling the gaps or “blind angles” or “black
swans” if efficient exchanges are organized ; this, inter alia, covers the fact that different national
approaches may address different types of ODDs (e.g. infrastructure design, weather, driving cultural
behaviors, driving direction).
DGITM/DMR/TUD GRVA-Scenarios-02
6-7 May 2024
- How to continuously improve the cost-benefit of using a scenario approach? e.g. by selecting scenarios
that combine a high representativeness of critical driving situations and an operational manageability for
implementing tests or, more generally, applying pass-fail criteria to systems’ responses ;
- How to avoid potential discrepancies in national practices in using scenario approaches to validate
automated systems? The challenge is to balance justified national specificities (e.g. based on national
features of local ODDs and driving behaviors, or expected behaviors regarding traffic rules) and the need
for the industry to optimize the costs of validation in multiple regional or national markets.
At first, these possible value-added of a more cooperative, collaborative or coordinated approach on
scenarios would raise interoperability issues to overcome to ease this cooperation (up to scenario-sharing
options?), between type-approval authorities, regulators, and the industry. Interoperability means to open
room for possible harmonized tools, practices or procedures stakeholders are eager to share, while some
others aspects remain out of the scope as competitive.
Second, the question of sharing lists of scenarios could be raised, for the seek of a better coverage and
benefiting from deployment in which new scenarios emerge from in-service monitoring and reporting.
Finally, as several scenario catalogues will exist and be used by various stakeholders, the question of
assessing “performance” of scenario catalogues will likely arise; assessing a scenario catalogue might refer
to define indicators and thresholds for different safety-validation objectives.
2- State of play for a possible approach
A lot has been done considering scenario taxonomy and guiding principles for scenario description,
generation and their increasing role within test procedures.
The scenario approach is thus well introduced in the ADS integrated document as “the guidelines recommend
the development of a scenario catalogue for use across five validation pillars”.
Moreover considering the EU-approach to regulating fully automated vehicles, scenarios are at the core of
the ADS regulation (EU) 2022/1426: scenario definition, guiding principles and concepts, scenario testing
(simulation and real testing). The interpretation document published by the EC-JRC (February 2024) gives
additional guidance for scenario generation (Appendix 2).
Finally when considering industrial and research initiatives among scenarios, it can be seen that various
stakeholders have launched their own platform to collect scenarios: ADScenes (FR industrial), MOSAR (FR
research), SafetyPool (UK), SAKURA (JP), PEGASUS (DE), StreetWise (NL), Fortellix (IL).
Joint efforts have been gathered on scenarios, especially concerning scenario taxonomy, and generation both
at the international level and at the European level, than at ISO level (3450X corpus).
These guidance illustrate the importance to combine two pillars in building a scenario catalogue: scenario
from data-based (accident and real world) sources and from knowledge-based (risk analysis) sources.
Nonetheless, the importance of the combinatory-based pillars remains to be fully addressed, though
introduced in the EC-JRC interpretation document from February 2024. The living principle of the scenario-
based approached relies in its ability to continually grow thanks to new descriptors (attributes) or axis,
building new scenarios.
In that perspective, the ability to share these augmented variables within both scenario catalogues and
scenario databases might be crucial in the line of deployments first, monitoring then.
As a complement, guiding principles of transparency, traceability and explicability of safety assessment
methods depends on the scenario-based approach itself.
These principles raise some other questions supporting proportionality of scenario use as for example moving
from generic ODDs to concrete deployment environments, moving from low to high severity and low to high
exposure, taking into account the “duty of care” to immediate crash avoidance in expected responses.
DGITM/DMR/TUD GRVA-Scenarios-02
6-7 May 2024
3- Draft agenda
It is proposed that discussions around cooperation, collaboration or coordination of scenarios-based
approaches and the management of scenario catalogues or databases would be structured along the
following scope and questions:
a) Is there a need for a UNECE catalogue?
• What requirements does a UNECE catalogue need to fulfil? / How would a UNECE catalogue be used?
• How would a UNECE catalogue interact with other national catalogues?
• How would a UNECE catalogue be produced and maintained?
• How do we fulfil these requirements?
• What will be the governance, including stakeholders, rights, responsibilities, etc.
b) How to ensure scenario database interoperability?
Once this is clearly defined, the workshop on scenarios intends to gather interesting stakeholders working
on scenarios and refers to a global “cooperation or coordination” around scenario activities, such as different
management / sharing / access options to scenario databases. It then raises the second question: what does
“cooperation or coordination” refer to?
Under this more flexible concept of cooperation – coordination, could be considered, inter alia:
• Common / inter-operable list of descriptors
• Minimum list of metadata
• Mutual access to collected scenarios (scene, hazards, responses)
• Mutual access to generation / combination / selection modules
• Mutual access to qualified scenarios for testing purposes
• Common access rules (e.g. special rights for regulators? Accident department)
• Common pricing principles (e.g. cost-orientation)
• Standardized interfaces / APIs
• References for quality levels (e.g. completeness, traceability, data sources’ consistency / quality)
• Classification of existing databases through common criteria
c) Is there consideration needed of the following scenario topics?
The proposed scope (enlarged from the initial concept of “UN scenario database”) would be
• Scenario definitions / scenario generation
o List of descriptors in close collaboration with ODD descriptors (scene, hazards, (response?),
other scenario attributes as for example exposure and severity) and the implementation of
the combinatory-based approach on OEDR elements
o Scenarios themselves (projected on descriptors, metadata as for example scenario sources,
use case)
• Scenario selection
• Test procedures implementing scenarios (including pass/fail criteria)
• More theoretically generation / combination / selection modules
• Governance features (requirements for database feeding, access rules and reuse conditions, prices)
