Test Set

Go up to Terms and Definitions

A Test Set groups related Test Objects for a defined Test Purpose.

As a Sequenced Test Object, a Test Set can contain or reference a Test Sequence. The Test Sequence establishes the order and progression of the Sequence Steps associated with the Test Set.

A Test Set can group:

• Other Test Sets
• Test Cases
• Test Steps
• Test Executables
• Test Items
• Test Environments
• Other related Test Objects

A Test Set can organize Test Objects according to:

• A capability
• A requirement
• A feature
• A behavior
• A scenario
• A risk
• A Test Item
• A System Under Test
• A Test Environment
• A test category
• A test phase
• A validation objective
• An Implementation Profile

A Test Set can identify:

• Its identity
• Its name
• Its Test Purpose
• Its version
• Its member Test Objects
• Its Test Sequence
• Its applicable Test Items
• Its applicable Test Environments
• Its applicable Governance Policies
• Its Provenance
• Its Traceability

A Test Set can be reused within multiple Test Definitions when its identity, version, membership, Test Sequence, and applicable conditions remain controlled.

Sequenced Test Object that groups related Test Objects for a defined Test Purpose

Adapted from:

• OMG Test Information Interchange Format (TestIF), Version 1.0 Beta 3, Section 7.3.15, TestSet
• Test Object
• Sequenced Test Object
• Test Case
• Test Sequence
• DIDO Reference Implementation Conceptual Model
• DIDO-TE draft Requirements Register

OMG TestIF defines TestSet as a SequencedTestObject that can serve as a container for multiple Test Sets, Test Cases, and related Test Objects.

The DIDO-TE definition preserves the TestIF grouping function while expressing the distinguishing characteristic without defining Test Set in terms of itself.

Every Test Set is a Sequenced Test Object and therefore a Test Object.

A Test Set can contain or reference another Test Set. Each nested Test Set retains its own identity, version, Test Purpose, membership, Provenance, and Traceability.

A Test Set differs from a Test Case:

• A Test Set groups related Test Objects
• A Test Case identifies the Test Steps and Expected Results used to verify something

A Test Set differs from a Node Set:

• A Test Set groups Test Objects
• A Node Set groups Nodes for a defined purpose

A Test Set differs from a Test Plan:

• A Test Set groups Test Objects
• A Test Plan identifies the objectives, scope, resources, responsibilities, schedule, and approach governing planned testing

A Test Set differs from a Test Definition:

• A Test Set provides an identified grouping of Test Objects
• A Test Definition specifies the controlled information required to conduct and evaluate a Test

A Test Set does not need to contain copies of its member Test Objects. It can reference independently identified and versioned Test Objects.

A change to the membership, Test Purpose, Test Sequence, or applicable conditions of a Test Set can create a new Test Set version when the change affects Test Execution, reproducibility, comparison, or validation.

A DIDO-TE interoperability Test Set groups:

• A Test Case for transaction submission
• A Test Case for transaction distribution
• A Test Case for state-transition validation
• A Test Case for state convergence
• A Test Case for recovery after a Node interruption

Each Test Case references the applicable:

• Test Items
• Test Environment
• Test Steps
• Test Executables
• Expected Results

During Test Execution, the Test Run records the Test Results associated with the Test Cases and other executed Test Objects in the Test Set.