Test Run State

Go up to Terms and Definitions

A Test Run State identifies the lifecycle condition of a Test Run at a particular time.

A Test Run changes from one Test Run State to another in response to lifecycle events occurring during Test Execution.

A Test Run State can identify whether the Test Run:

• Has not started
• Is ready to start
• Is running
• Is paused
• Completed its defined execution
• Terminated without completing its defined execution

The standard Test Run States are:

• Not Started
• Ready
• Running
• Paused
• Completed
• Terminated

Not Started identifies a Test Run that exists but has not begun Test Execution.

Ready identifies a Test Run for which the applicable prerequisites have been satisfied and which can begin Test Execution.

Running identifies a Test Run actively performing its applicable Test Sequence, Sequence Steps, Test Steps, or Test Executables.

Paused identifies a Test Run whose Test Execution has temporarily stopped and can resume according to the applicable control rules.

Completed identifies a Test Run that satisfied its defined completion condition.

Terminated identifies a Test Run that stopped without satisfying its defined completion condition.

A Test Run in the Terminated state has an applicable Test Run Termination Reason.

A Test Run State can identify:

• Its name
• Its description
• Its entry condition
• Its exit condition
• Its permitted preceding states
• Its permitted following states
• Its applicable lifecycle events
• Its applicable Governance Policies
• Its applicable Authority
• Its recorded time
• Its Evidence
• Its Provenance
• Its Traceability

A Test Run record can preserve each Test Run State transition, including:

• The preceding Test Run State
• The following Test Run State
• The event causing the transition
• The time of the transition
• The Actor requesting or causing the transition
• The applicable Governance Policies
• The result of the transition
• The supporting Evidence

lifecycle condition of a Test Run at a particular time

Adapted from:

• OMG Test Information Interchange Format (TestIF), Version 1.0
• Test Run
• Test Execution
• DIDO Reference Implementation Conceptual Model
• DIDO-TE draft Requirements Register

OMG TestIF provides Test Run, Test Result, and Verdict concepts for exchanging test information and execution results.

OMG TestIF does not prescribe the DIDO-TE Test Run lifecycle states defined on this page. This definition introduces Test Run State for DIDO-TE operational control, lifecycle management, Evidence, Provenance, and Traceability.

A Test Run has exactly one current Test Run State at a particular time.

The current Test Run State does not replace the history of Test Run State transitions. A Test Run record can preserve every state occupied by the Test Run and every transition among those states.

A Test Run State differs from a Test Run Termination Reason:

• A Test Run State identifies the lifecycle condition of a Test Run
• A Test Run Termination Reason identifies why a Test Run entered the Terminated state

A Test Run State differs from a Verdict:

• A Test Run State identifies the lifecycle condition of a Test Run
• A Verdict expresses an evaluation of a Test Result

A Test Run can enter the Completed state and still produce a Test Result assigned a failing Verdict.

A Test Run can enter the Terminated state without producing sufficient Test Results to assign a conclusive Verdict.

The Completed state means the Test Run satisfied its defined completion condition. It does not mean:

• Every Test Result received a passing Verdict
• The tested system satisfied every applicable Requirement
• A favorable Validation Decision was made
• No errors or warnings occurred

The Terminated state means the Test Run stopped without satisfying its defined completion condition. The applicable Test Run Termination Reason identifies why it stopped.

The Paused state differs from the Terminated state because a paused Test Run remains eligible to resume.

The applicable Governance Policies determine:

• Which Actors can initiate a Test Run
• Which Actors can pause or resume a Test Run
• Which Actors can cancel a Test Run
• Which Test Run State transitions are permitted
• Which Evidence must be preserved for each transition

An authorized User creates a Test Run for a selected Test Definition.

The Test Run progresses through the following Test Run States:

• Not Started when DIDO-TE creates the Test Run record
• Ready after DIDO-TE confirms the applicable Starting Conditions
• Running when Test Execution begins
• Paused when an authorized User requests a temporary suspension
• Running when the authorized User resumes Test Execution
• Completed when the Test Run satisfies its defined completion condition

DIDO-TE records each transition and preserves the applicable Actor, time, reason, Governance Policies, and Evidence.