ASAM Test Specification Study Group Report 2022

Executive Summary

Although sometimes it may feel like we have a frustratingly limited number of ways to steer a vehicle through traffic, the reality is that driving is tremendously complex, with an infinite number of possible scenarios – particularly when it comes to testing requirements. One question the current developments will soon pose is: who exactly is behind the steering wheel? Today, we see the adoption of SAE Level 0 to Level 2 advanced driver assistance systems (ADAS) throughout the industry, including features such as Brake Assist, Lane Keeping Assist, Electronic Stability Control, Blind Spot Indication, or Parking Assist. While these features do not eliminate the role of the driver, the interaction with the environment increasingly transfers from driver to vehicle. In other words, the transition from assisted driving to fully automated driving is on the horizon. Consequently, the automotive industry is undergoing the biggest changes in its history. This paradigm shift affects everyone and brings opportunities and challenges to the industry.

Levels of Autonomy and the Difficulty of Proving Safety
The future of transportation promises to make life safer and more mobile for everyone, with positive economic results. But to realize that promise, it is first necessary to test new vehicles and every part of their architecture, especially as the parts become smarter and highly complex. However, the increased complexity necessitates a radical change of test methods and new concepts for comprehensive vehicle validation in both the physical and the virtual world. Still, many players appear to have neglected to consider the best path for developing and implementing suitable strategies. Without toolkits independent of manufacturers or domains, and without intensive exchange across all subsectors, highly autonomous driving simply cannot be realized. We are on the verge of ushering in a new era of transportation safety through autonomous mobility, but this shift raises big questions. What constitutes state-of-the-art testing of driving functions in a rapidly evolving environment? How will the automotive industry a) consider best practices for testing and safe deployment while b) at the same time integrating new technologies like Artificial Intelligence and c) complying with regulations that are not yet developed? In February 2021, the UNECE (United Nations Economic Commission for Europe) presented the New Assessment/Test Method for Automated Driving (NATM) – a framework and potential game changer. Its multi-pillar approach considers the topic’s high level of complexity and poses new opportunities for more comprehensive requirement sets as a focus of technical services. However, these still need to be created and standardized.

Why the Joint Effort?
To expedite this process, a transdisciplinary network of experts was formed, the ASAM Test Specification Study Group, including representatives of technical services, software providers, manufacturers and OEMS, engineering and test data management professionals.
Under the banner of ASAM e.V. experts have collaborated for more than twenty years in order to create standards for the development and testing of automotive electrical/electronic systems. They succeeded in reducing costs and efforts at OEMs and Tier-1 companies for the development, maintenance, and administration of development tool chains. Currently the ASAM portfolio comprises more than 30 trusted standards that are applied in automotive development across the globe. While this means that many pieces of the puzzle when it comes to standardized scenario-based testing workflows are being tackled, the overall interplay between those pieces had not been examined in such close detail. The following report documents the results of this new joint effort and aims to define a valid basis for follow-up activities and projects.

Main goals of the ASAM Test Specification Study Group project:

• Provide overview of test methods in the field of ADAS/AD
• Develop a potential basis for future testing, the Test Strategy Blueprint
• Detailed use cases for the implementation of a test strategy
• Alignment with current standardizations
• Provide recommendations for stakeholders and proposals for further standardization activities

A New Strategy Blueprint
Based on the experience of the many experts working together during the course of the ASAM Test Specification Study Group project, a blueprint to meet the challenges of testing has been developed. Test strategies are becoming more complex, but this blueprint makes it possible to set a sensible starting point. A holistic best practice that can be tailored according to the specific requirements of manufacturing and other projects, but one that meets regulatory, legal, and technical requirements. This blueprint is to be understood as an invitation. An invitation to use, develop, and critically engage with. An opportunity to work together on the challenges of an increasingly complex automotive industry.
The implementation of new testing strategies is an intricate task. Therefore, we defined concrete workflows and identified the need for a uniform test data management. In addition, we provide initial solutions so that users are not left alone. Despite not being able to analyze and highlight all aspects, we were able to define concrete recommendations for action and follow-up activities. Current test strategies, which are often heterogeneous and have grown over several vehicle generations, must be viewed holistically and also used during vehicle approval. The various test procedures and test environments highlighted in the blueprint can also be clearly anchored in the data-driven process. It is important to understand that the phases are no longer strictly separated from one another, but that transitions are smooth and continuous. Iterations and changes between the phases occur constantly and merge into one another. In the following chart you can see a possible and reasonable combination to fulfill test coverage for a software-centric and (partially) autonomous vehicle, which is sufficient for release and homologation.

The derived blueprint, exemplifying test coverage for a software-centric vehicle

The Vision of Seamless Exchange
The following chapters and sub-sections elaborate on the current standardization landscape and possible future developments, describe relevant use cases and how the new test strategy blueprint can be used. The report aims to provide overview and to additionally clarify the role of data-driven development and the importance of test data management. The final chapter summarizes the study group’s findings, deducing recommendations for further action. Competition stimulates progress. If a specific technology is not effective for its users, however, competition loses its benefits. It leads to incompatible systems rather than better products. That is one reason why ASAM pursues a vision of the free interconnection of development process chain tools and the seamless exchange of data. The combined efforts of the ASM Test Specification Study Group bear witness to the fertility of this concept.