public:99_annexes:annex-b-terms-and-definitions:i:incomplete_test_run

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
public:99_annexes:annex-b-terms-and-definitions:i:incomplete_test_run [2026/09/12 17:42] – removed - external edit (Unknown date) 127.0.0.1public:99_annexes:annex-b-terms-and-definitions:i:incomplete_test_run [2026/09/12 17:43] (current) – ↷ Links adapted because of a move operation nick_dido
Line 1: Line 1:
 +====== Incomplete Test Run ======
  
 +[[public:99_annexes:annex-b-terms-and-definitions:start|Go up to Terms and Definitions]]
 +
 +===== Discussion =====
 +
 +An Incomplete Test Run is a [[public:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] that terminated without satisfying its defined completion condition.
 +
 +An Incomplete Test Run has:
 +
 +  * A Terminated [[public:99_annexes:annex-b-terms-and-definitions:t:test_run_state|Test Run State]]
 +  * A [[public:99_annexes:annex-b-terms-and-definitions:t:test_run_termination_reason|Test Run Termination Reason]]
 +  * An unsatisfied completion condition
 +
 +An Incomplete Test Run can have the following Test Run Termination Reason:
 +
 +  * Failed
 +  * Interrupted
 +  * Timed Out
 +  * Canceled
 +
 +An Incomplete Test Run can preserve valid information produced before termination, including:
 +
 +  * Completed [[public:99_annexes:annex-b-terms-and-definitions:s:sequence_step|Sequence Steps]]
 +  * Completed [[public:99_annexes:annex-b-terms-and-definitions:t:test_step|Test Steps]]
 +  * Performed [[public:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executables]]
 +  * Applied [[public:99_annexes:annex-b-terms-and-definitions:t:test_argument_value|Test Argument Values]]
 +  * Observations
 +  * Measurements
 +  * Events
 +  * Outputs
 +  * [[public:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:v:verdict|Verdicts]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:p:provenance|Provenance]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:t:traceability|Traceability]]
 +
 +The applicable [[public:99_annexes:annex-b-terms-and-definitions:a:acceptance_criteria|Acceptance Criteria]] determine whether information produced by an Incomplete Test Run can be used for:
 +
 +  * Analysis
 +  * Diagnosis
 +  * Comparison
 +  * Regression analysis
 +  * Baseline establishment
 +  * [[public:99_annexes:annex-b-terms-and-definitions:v:validation|Validation]]
 +  * A [[public:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]]
 +  * Another defined purpose
 +
 +===== Definition =====
 +
 +//[[public:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] that entered the Terminated [[public:99_annexes:annex-b-terms-and-definitions:t:test_run_state|Test Run State]] without satisfying its defined completion condition//
 +
 +===== Source =====
 +
 +Adapted from:
 +
 +  * [[https://www.omg.org/spec/TestIF/1.0/PDF|OMG Test Information Interchange Format (TestIF), Version 1.0]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:t:test_execution|Test Execution]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:t:test_run_state|Test Run State]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:t:test_run_termination_reason|Test Run Termination Reason]]
 +  * DIDO Reference Implementation Conceptual Model
 +  * DIDO-TE draft Requirements Register
 +
 +OMG TestIF provides Test Run, Test Result, and Verdict concepts but does not define Incomplete Test Run as a separate TestIF model element.
 +
 +This definition introduces Incomplete Test Run to distinguish a Test Run that failed to satisfy its completion condition from a completed Test Run producing a Test Result assigned a failing Verdict.
 +
 +===== Note =====
 +
 +An Incomplete Test Run is a Test Run in the Terminated Test Run State.
 +
 +A Test Run that is Not Started, Ready, Running, or Paused has not yet completed, but it is not classified as an Incomplete Test Run because its [[public:99_annexes:annex-b-terms-and-definitions:t:test_execution|Test Execution]] has not terminated.
 +
 +A Test Run becomes an Incomplete Test Run when:
 +
 +  * Test Execution stops
 +  * The Test Run enters the Terminated Test Run State
 +  * The defined completion condition remains unsatisfied
 +
 +An Incomplete Test Run differs from a completed Test Run assigned a failing Verdict:
 +
 +  * An Incomplete Test Run did not satisfy its completion condition
 +  * A completed Test Run assigned a failing Verdict satisfied its completion condition and produced a Test Result that did not satisfy the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:expected_result|Expected Result]] or [[public:99_annexes:annex-b-terms-and-definitions:v:validation_criteria|Validation Criteria]]
 +
 +An Incomplete Test Run does not imply that every Test Result produced before termination is invalid.
 +
 +The applicable Acceptance Criteria determine:
 +
 +  * Which partial Test Results remain valid
 +  * Which Evidence remains usable
 +  * Whether the Test Run can resume
 +  * Whether DIDO-TE must create a new Test Run
 +  * Whether the Test Run can participate in a comparison
 +  * Whether the Test Run can contribute to a [[public:99_annexes:annex-b-terms-and-definitions:b:baseline|Baseline]]
 +  * Whether the Test Run can support a Validation Decision
 +  * Whether the Test Run must be repeated
 +
 +A paused Test Run is not an Incomplete Test Run because it remains eligible to resume.
 +
 +A canceled Test Run is an Incomplete Test Run when cancellation causes it to enter the Terminated state before satisfying its completion condition.
 +
 +A timed-out Test Run is an Incomplete Test Run when expiration of an applicable [[public:99_annexes:annex-b-terms-and-definitions:t:timeout|Timeout]] causes it to enter the Terminated state before satisfying its completion condition.
 +
 +===== Example =====
 +
 +A Test Run contains ten [[public:99_annexes:annex-b-terms-and-definitions:s:sequence_step|Sequence Steps]].
 +
 +The Test Run successfully completes the first six Sequence Steps. During the seventh Sequence Step, an [[public:99_annexes:annex-b-terms-and-definitions:e:execution_facility|Execution Facility]] becomes unavailable and Test Execution cannot continue.
 +
 +DIDO-TE:
 +
 +  * Stops Test Execution
 +  * Changes the Test Run State from **Running** to **Terminated**
 +  * Assigns **Interrupted** as the Test Run Termination Reason
 +  * Classifies the Test Run as an Incomplete Test Run
 +  * Preserves the Test Results produced by the first six Sequence Steps
 +  * Records the affected Sequence Step
 +  * Records the unavailable Execution Facility
 +  * Preserves the supporting Evidence
 +  * Maintains Provenance and Traceability
 +
 +The applicable Acceptance Criteria determine whether the preserved Test Results can be used and whether DIDO-TE must repeat the entire Test Run or resume from an approved Sequence Step.
 +
 +----
 +
 +<WRAP centeralign>
 +© 2026 Dido Solutions, Inc. and Jackrabbit Consulting, Inc.
 +</WRAP>