Automotive Industry Insights

The trustworthiness of test environments

ISO 26262 Tool Qualification
As an automation system has control over the major actuations of the vehicle – longitudinal and lateral control – its decisions are safety critical. Thus, automation systems are subject to ISO 26262. As ISO 26262 affects requirements for testing and the use of SW tools that are applied to cover objectives from the standard. Therefore, all parts of the test tool chain used to verify or validate safety-critical decisions will be subject to tool qualification. This means the tool use cases need to be specified and compliance with these specifications has to be shown in agreement with the provisions in ISO 26262.

Model Validation
Similar to the in-vehicle automation function that has functional safety and SOTIF aspects, models used for verification and validations also need to be executed with integrity and need to provide a certain minimum performance in terms of agreement with the real world (e.g. ISO 11010). In the previous paragraph on proving ground tests, it was already outlined that there is the need to support the lower-level in-the-loop tests in terms of validation of the deployed models. By nature, a model is a simplification of the real world. The trick with modeling is thus to remove complexity that is not relevant for the process of interest, keeping the relevant parts sufficiently realistic. It needs to be confirmed whether this abstraction process has been successful. This confirmation is typically done by validating the models. This means the model results are confronted with results obtained in real-world experiments. Only with such input on the trustworthiness of the tool chain can the results be valued correctly during homologation.

Test Environment Adequacy and Fit-for-Purpose Checks
Verification and validation strategies have a full product life cycle impact. Thus, tool environments have to face requirements related to reliability, long-term support, and reproducibility of results. This can only be accomplished if the test environments are used based on a solid quality management system and the recurrent prove of suitability (hardware calibrations).

Risk Assessments, Expert Knowledge, Systems Designs
It has been discussed above that testing for automated driving is strongly interrelated with other development activities. This implies that a test strategy blueprint cannot be a static, stand-alone document. Instead, it needs to adapt to other sources. Such sources are risk assessment methods to be deployed, available expert knowledge on the particular automation system, and also the choice of system architecture and design.