Test Run Termination Reason

Go up to Terms and Definitions

A Test Run Termination Reason identifies why a Test Run entered the Terminated Test Run State.

A Test Run enters the Terminated Test Run State when it stops without satisfying its defined completion condition.

The standard Test Run Termination Reasons are:

• Failed
• Interrupted
• Timed Out
• Canceled

Failed identifies termination caused by an error or failure that prevents continued Test Execution.

Interrupted identifies termination caused by an unplanned event external to the normal Test Execution progression.

Timed Out identifies termination caused by expiration of an applicable Timeout before the applicable completion condition was satisfied.

Canceled identifies termination caused by an authorized cancellation request or applicable Governance Policy.

A Test Run Termination Reason can identify:

• Its name
• Its description
• Its classification
• The event causing termination
• The time of termination
• The Actor requesting or causing termination
• The affected Test Step
• The affected Sequence Step
• The affected Test Executable
• The applicable Timeout
• The applicable Governance Policies
• The applicable error or failure information
• The resulting Test Results
• Its Evidence
• Its Provenance
• Its Traceability

A Test Run Termination Reason does not determine the Verdict assigned to a Test Result.

reason a Test Run entered the Terminated Test Run State

Adapted from:

• OMG Test Information Interchange Format (TestIF), Version 1.0
• Test Run
• Test Execution
• Timeout
• DIDO Reference Implementation Conceptual Model
• DIDO-TE draft Requirements Register

OMG TestIF provides Test Run, Test Result, and Verdict concepts but does not prescribe the DIDO-TE Test Run termination-reason classification defined on this page.

This definition introduces Test Run Termination Reason to distinguish why Test Execution stopped from the lifecycle state of the Test Run and from the evaluation of its Test Results.

A Test Run Termination Reason applies when a Test Run enters the Terminated Test Run State.

A Test Run that enters the Completed Test Run State does not require a Test Run Termination Reason because it satisfied its defined completion condition.

A Test Run Termination Reason differs from a Test Run State:

• A Test Run State identifies the lifecycle condition of the Test Run
• A Test Run Termination Reason identifies why the Test Run entered the Terminated state

A Test Run Termination Reason differs from a Verdict:

• A Test Run Termination Reason classifies why Test Execution stopped
• A Verdict evaluates a Test Result

The termination reason Failed does not mean a Test Result received a failing Verdict.

A Test Run can:

• Complete normally and produce a Test Result assigned a failing Verdict
• Terminate because of a failure and produce no conclusive Verdict
• Terminate because of a failure while preserving valid Test Results produced before termination
• Be canceled even though every Test Result produced before cancellation received a passing Verdict
• Time out because an expected event did not occur within an applicable Timeout

The termination reason Interrupted applies to an unplanned event that stops Test Execution. Examples can include:

• Loss of an Operational Resource
• Loss of communication
• Loss of power
• Failure of an external system
• Unplanned termination of an Execution Facility
• Another external event preventing continued Test Execution

The termination reason Canceled applies when an authorized Actor or Governance Policy intentionally stops the Test Run.

The applicable Governance Policies or Acceptance Criteria should determine how to classify a termination when more than one reason appears applicable.

The Test Run record should preserve the immediate termination reason and can also preserve contributing events, errors, failures, and conditions.

A Test Run performs a Test Sequence requiring a Node to return a response within 30 seconds.

The Node does not return the response before the applicable Timeout expires.

DIDO-TE:

• Stops the applicable Test Execution
• Changes the Test Run State from Running to Terminated
• Assigns Timed Out as the Test Run Termination Reason
• Records the applicable Timeout
• Preserves the Test Results produced before termination
• Preserves the supporting Evidence
• Maintains Traceability to the affected Sequence Step and Test Step

The Test Run is also an Incomplete Test Run because it terminated without satisfying its defined completion condition.