Terms and Definitions

Test Method

The term “test method” is not defined in any of the common standards like the ISTQB glossary or ISO 29119, but there are some hints how the term could be used. The American Society for Testing and Materials defines “test method” as :

“… a method for a test in science or engineering, such as a physical test, chemical test, or statistical test. It is a definitive procedure that produces a test result. In order to ensure accurate and relevant test results, a test method should be ‘explicit, unambiguous, and experimentally feasible,’ as well as effective and reproducible.”

Similarities to other standards like software methodology are object-oriented, sequential, data-driven, and agile software development methods. In the context of highly automated driving we speak also about the PEGASUS Method for Assessment of Highly Automated Driving Function (HAD-F) propagated from the PEGASUS project (https://www.pegasusprojekt.de/en/).

A test method is any procedure that fulfils test goals and defines, for example, applicable test techniques or practices as a part of this specific method. Following this idea, requirements-based testing, for example, is a test practice, but not a test method in itself. There are many methods to conduct requirements-based tests to obtain test results and fulfil test goals.

A test method may consist of

• Choice of test practices (requirements-based testing, scenario-based testing, model-based testing)
• Test level (system test level, integration test level)
• Test environments (HIL, SIL, MIL, VIL, proving ground, real road testing)
• Test techniques (equivalence partitioning, state-based testing, failure injection test (?)
• Applicable procedures/processes (postprocessing, risk assessment)
• Choice of coverage criteria

So, a test method represents a partial implementation of the test strategy for an aspect of the test item to be tested.