Test Data Management
Data Modeling Based on Test Strategy and Use Cases
To describe test data management aspects in ADAS/AD function development we propose a first high-level domain model as a basis for furthering modeling in future standardization efforts. The Test Strategy Blueprint and the according Use Cases defined their serve as bases as these are valid for the initial development process of ADAS functions until homologation.
To develop the high-level ADAS-specific domain model as the basis for proper test data management the use cases were investigated to find important artifacts. These could be test inputs, test outputs, or information necessary to assist the test preparation. Based on these findings the group developed an abstract “ADAS/AD domain” model, bearing the aspect of test data management in mind. The artifacts were displayed graphically and grouped according to their relation to one another.
The diagram distinguishes graphically between information which is necessary to (re-)conduct a test run, marked in green, and information which is created during or after a test run (marked in orange). It is based on the assumption that test specifications are derived from requirements. A description of the displayed artifacts is presented below.
| Artifact | Description |
| Test case | The different abstraction levels of a test case (logical down to concrete) are represented in the diagram through a reference to itself |
| Bus data | Describes in this context the recorded bus signal which was sent during the test execution |
| Test run | The execution of a test suite on a specific version of the test object (ISTQB) |
| Coordinates | In this context describes the movement of one or several test objects through a relative (e.g. proving ground) or global (GPS) coordinate system |
| Coverage data | The degree to which specified coverage items have been determined or have been exercised by a test suite expressed as a percentage (ISTQB) |
| Documents/reports | Final reports in different formats might be stored attached to tests, as it has been described in several testing workflows |
| Events | Singular moment in time at which some perceptible phenomenological change (energy, matter, or information) occurs at a certain place (ISO 21448, ISO 24765:2017, 3.1484).
Trigger event: |
| Image streams | Includes image-driven sensor data, e.g. lidar, radar, video data |
| Label | Set of labels which describe a scenario and the objects in the scenario (openLabel) |
| Operational design domain | Description of the specific operation domain(s) in which an automated function or system is designed to properly operate, including but not limited to roadway types, speed range, and environmental conditions |
| Requirement | A condition or capability needed by a stakeholder that must be met or possessed by a system, product, service, result, or component to satisfy a contract, standard, or specification. Or another formally imposed document to solve a problem or achieve an objective (ISO 24765:2017, 3.3431) |
| Road network | Description of the geometry of roads, lanes, and objects, such as road marks on the road, as well as features along the roads, like signals (ASAM OpenDRIVE®) |
| Road surface | Description of the road surface used in the road network description |
| Scenario description | Reference to chapter 5.3.9.1 Scenario Definition |
| Sensor data | In this context it relates to the “classical” sensor data related to physical dimensions (e.g. pressure, acceleration, voltage, etc.) as it is used in non-ADAS/AD test fields |
| Test description | Includes all descriptive elements which describe the execution of a concrete test. The test description includes all information about the used test sequence, test equipment, and unit under test |
| Test result | Indication of whether or not a specific test case has passed or failed, i.e. if the actual result observed as test item output corresponds to the expected result or if deviations were observed |
| Test sequence | Contains information about which sequence of steps has been processed during testing (ASAM ODS). In the reference workflows the following related artifacts have been found:
In a future standardization project the description of these and further related information shall be standardized |
| Test equipment | Contains information about which equipment has been used (ASAM ODS). In the reference workflows the following related artifacts have been found:
In a future standardization project, the description of these and further related equipment shall be standardized |
| Test suite/campaign | A test suite is a group of test cases. A test campaign is the realization of an entire test strategy |
| Unit under test | Contains information about whether what has been tested (ASAM ODS) matches the test object in ISO. In the reference workflows the following related artifacts have been found:
|
Chapter
Share
Share and discuss this content with your network. Thank you!
Contact
ASAM e.V.
Altlaufstraße 40
85635 Höhenkirchen
Phone: +49 8102 806160
Email: info@asam.net
