====== Incomplete Test Run ====== [[dido:99_annexes:annex-b-terms-and-definitions:start|Go up to Terms and Definitions]] ===== Discussion ===== An Incomplete Test Run is a [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] that terminated without satisfying its defined completion condition. An Incomplete Test Run has: * A Terminated [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_state|Test Run State]] * A [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_termination_reason|Test Run Termination Reason]] * An unsatisfied completion condition An Incomplete Test Run can have the following Test Run Termination Reason: * Failed * Interrupted * Timed Out * Canceled An Incomplete Test Run can preserve valid information produced before termination, including: * Completed [[dido:99_annexes:annex-b-terms-and-definitions:s:sequence_step|Sequence Steps]] * Completed [[dido:99_annexes:annex-b-terms-and-definitions:t:test_step|Test Steps]] * Performed [[dido:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executables]] * Applied [[dido:99_annexes:annex-b-terms-and-definitions:t:test_argument_value|Test Argument Values]] * Observations * Measurements * Events * Outputs * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]] * [[dido:99_annexes:annex-b-terms-and-definitions:v:verdict|Verdicts]] * [[dido:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]] * [[dido:99_annexes:annex-b-terms-and-definitions:p:provenance|Provenance]] * [[dido:99_annexes:annex-b-terms-and-definitions:t:traceability|Traceability]] The applicable [[dido:99_annexes:annex-b-terms-and-definitions:a:acceptance_criteria|Acceptance Criteria]] determine whether information produced by an Incomplete Test Run can be used for: * Analysis * Diagnosis * Comparison * Regression analysis * Baseline establishment * [[dido:99_annexes:annex-b-terms-and-definitions:v:validation|Validation]] * A [[dido:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]] * Another defined purpose ===== Definition ===== //[[dido:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] that entered the Terminated [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_state|Test Run State]] without satisfying its defined completion condition// ===== Source ===== Adapted from: * [[https://www.omg.org/spec/TestIF/1.0/PDF|OMG Test Information Interchange Format (TestIF), Version 1.0]] * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution|Test Execution]] * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_state|Test Run State]] * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_termination_reason|Test Run Termination Reason]] * DIDO Reference Implementation Conceptual Model * DIDO-TE draft Requirements Register OMG TestIF provides Test Run, Test Result, and Verdict concepts but does not define Incomplete Test Run as a separate TestIF model element. This definition introduces Incomplete Test Run to distinguish a Test Run that failed to satisfy its completion condition from a completed Test Run producing a Test Result assigned a failing Verdict. ===== Note ===== An Incomplete Test Run is a Test Run in the Terminated Test Run State. A Test Run that is Not Started, Ready, Running, or Paused has not yet completed, but it is not classified as an Incomplete Test Run because its [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution|Test Execution]] has not terminated. A Test Run becomes an Incomplete Test Run when: * Test Execution stops * The Test Run enters the Terminated Test Run State * The defined completion condition remains unsatisfied An Incomplete Test Run differs from a completed Test Run assigned a failing Verdict: * An Incomplete Test Run did not satisfy its completion condition * A completed Test Run assigned a failing Verdict satisfied its completion condition and produced a Test Result that did not satisfy the applicable [[dido:99_annexes:annex-b-terms-and-definitions:e:expected_result|Expected Result]] or [[dido:99_annexes:annex-b-terms-and-definitions:v:validation_criteria|Validation Criteria]] An Incomplete Test Run does not imply that every Test Result produced before termination is invalid. The applicable Acceptance Criteria determine: * Which partial Test Results remain valid * Which Evidence remains usable * Whether the Test Run can resume * Whether DIDO-TE must create a new Test Run * Whether the Test Run can participate in a comparison * Whether the Test Run can contribute to a [[dido:99_annexes:annex-b-terms-and-definitions:b:baseline|Baseline]] * Whether the Test Run can support a Validation Decision * Whether the Test Run must be repeated A paused Test Run is not an Incomplete Test Run because it remains eligible to resume. A canceled Test Run is an Incomplete Test Run when cancellation causes it to enter the Terminated state before satisfying its completion condition. A timed-out Test Run is an Incomplete Test Run when expiration of an applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:timeout|Timeout]] causes it to enter the Terminated state before satisfying its completion condition. ===== Example ===== A Test Run contains ten [[dido:99_annexes:annex-b-terms-and-definitions:s:sequence_step|Sequence Steps]]. The Test Run successfully completes the first six Sequence Steps. During the seventh Sequence Step, an [[dido:99_annexes:annex-b-terms-and-definitions:e:execution_facility|Execution Facility]] becomes unavailable and Test Execution cannot continue. DIDO-TE: * Stops Test Execution * Changes the Test Run State from **Running** to **Terminated** * Assigns **Interrupted** as the Test Run Termination Reason * Classifies the Test Run as an Incomplete Test Run * Preserves the Test Results produced by the first six Sequence Steps * Records the affected Sequence Step * Records the unavailable Execution Facility * Preserves the supporting Evidence * Maintains Provenance and Traceability The applicable Acceptance Criteria determine whether the preserved Test Results can be used and whether DIDO-TE must repeat the entire Test Run or resume from an approved Sequence Step. ---- © 2026 Dido Solutions, Inc. and Jackrabbit Consulting, Inc.