Recommendations

Recommendations

The ASAM Test Specification Study Group project examined relevant techniques and use cases for testing and homologation of software-centric vehicles and automated driving functions. Identifying relevant standards, potential workflows, their variants, and the interplay between these parts led to a documented set of use cases, as well as to a set of potential workflows to implement them. Based on these findings, the study group developed a blueprint for test strategies that are ready for use by stakeholders across the automotive industry and across development phases. Moreover, the test-specification study group conducted a thorough white spot analysis, pointing out workflow gaps to identify where to supplement existing standards. The following recommendations are therefore the product of a joint effort by participants representing industry practice from various angles. Their shared goal is to establish a common pool of knowledge for further research, exchange, improvement and alignment. The bigger picture? A vehicle on the road and safety for everyone.

The New Test Strategy Blueprint as a Starting Point for Future Collaboration
The ASAM Test Specification Study Group is introducing a blueprint through this publication designed as a first point of reference. It was comprehensively developed with the implicit aim to be used, extended, or changed according to technological progress and testing possibilities. The recommendations linked to it are first and foremost an invitation.

Recommendations for a New Test Specification Standard
Scenario-based testing is highly popular and relevant for the safety argumentation, especially for automated vehicles. However, it is very clear from the report that this is only one part of testing. For a holistic release of the vehicle, its software, and electronics, a holistic approach as described in the Test Strategy Blueprint is necessary. To ensure that the different test methods and test environments along this test strategy are applicable and used, and firstly to use synergies, secondly to simplify transitions, and thirdly to increase the reusability of artefacts in all these tests, the study group proposes a standardized interface between a scenario description and a test case. It is recommended that a working group be set up to define and specify this interface.

The following aspects should be addressed:

• Depending on the use case there are test cases without any scenario description or test cases using one or even multiple scenario descriptions
• Different scenario description languages/standards exist but they need to be able to be connected to tests in flexible and common way.
• For the test description, different languages/standards are applied.
• Features of the “standardized interface” are driven from the test perspective; the scenario may not need to “learn” something from the test and vice-versa. Testing “drives” the requirements on this interface and scenario descriptions implement such an interface (handshake).

It is important to emphasize that different standardization domains should not be mixed up by addressing them with a common standard. This holds especially true for the two domains of automotive testing and scenarios, where the above-mentioned interface between tests and scenarios should therefore separate the associated domains while also enabling their interaction.

Test Data Management as Basis for Release and Homologation
It has become clear in the course of the analysis that data management is necessary for the organization of the complex test landscape. In the course of the study, we were able to clearly demonstrate that the end-to-end testing and homologation of vehicles can be usefully supported with such test data management. However, we were not able to offer any analysis beyond the details of the workflows presented here. For example, topics such as traceability have not yet been sufficiently considered. The recommendation of the ASAM Test Specification Study Group is quite clear: to now follow up with a deep dive into the topic of data management and the associated (already existing) standards. For recommendations regarding specific of use cases or individual standards, please refer to the corresponding subsections.

Proposals for Global Alignment
As shown in the report, the map of testing is becoming increasingly complex and efforts to view these different activities holistically are also very heterogenous. This report serves as an invitation to analyze the different activities together and to look at them holistically. The clear recommendation is therefore to bring about a worldwide coordination of standardization in continuous follow-up activities. We currently see no tendency to reduce and unify standards and research projects. Therefore, our opinion is to invest more in alignments. The goal should be to answer the following question: How can standardization be managed globally in the future? How long will we be able to accept this uncontrolled spread?

The automotive landscape is evolving but so is how we collaborate. Let’s put it to the test.