public:99_annexes:annex-b-terms-and-definitions:t:test_execution

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
public:99_annexes:annex-b-terms-and-definitions:t:test_execution [2026/09/12 17:42] – removed - external edit (Unknown date) 127.0.0.1public:99_annexes:annex-b-terms-and-definitions:t:test_execution [2026/09/12 17:43] (current) – ↷ Links adapted because of a move operation nick_dido
Line 1: Line 1:
 +====== Test Execution ======
  
 +[[public:99_annexes:annex-b-terms-and-definitions:start|Go up to Terms and Definitions]]
 +
 +===== Discussion =====
 +
 +Test Execution is the performance of one or more [[public:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Objects]] within a [[public:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] to produce one or more [[public:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]].
 +
 +Test Execution can perform:
 +
 +  * A [[public:99_annexes:annex-b-terms-and-definitions:t:test_set|Test Set]]
 +  * A [[public:99_annexes:annex-b-terms-and-definitions:t:test_case|Test Case]]
 +  * A [[public:99_annexes:annex-b-terms-and-definitions:t:test_step|Test Step]]
 +  * A [[public:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executable]]
 +
 +Test Execution can include:
 +
 +  * Selecting the applicable Sequenced Test Objects
 +  * Establishing the applicable Test Environment
 +  * Confirming applicable preconditions
 +  * Selecting the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_item|Test Items]]
 +  * Selecting the applicable [[public:99_annexes:annex-b-terms-and-definitions:n:node_set|Node Sets]]
 +  * Selecting the applicable [[public:99_annexes:annex-b-terms-and-definitions:d:deployment_target|Deployment Targets]]
 +  * Resolving the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_argument|Test Arguments]]
 +  * Supplying the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_argument_value|Test Argument Values]]
 +  * Selecting the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:execution_facility|Execution Facilities]]
 +  * Retrieving the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:executable_artifact|Executable Artifacts]]
 +  * Interpreting or performing the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:executable_instruction|Executable Instructions]]
 +  * Observing the behavior of the Test Items
 +  * Measuring applicable conditions and outcomes
 +  * Comparing observed outcomes with [[public:99_annexes:annex-b-terms-and-definitions:e:expected_result|Expected Results]]
 +  * Assigning applicable [[public:99_annexes:annex-b-terms-and-definitions:v:verdict|Verdicts]]
 +  * Producing Test Results
 +  * Collecting [[public:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]]
 +  * Preserving [[public:99_annexes:annex-b-terms-and-definitions:p:provenance|Provenance]]
 +  * Maintaining [[public:99_annexes:annex-b-terms-and-definitions:t:traceability|Traceability]]
 +
 +A [[public:99_annexes:annex-b-terms-and-definitions:t:test_sequence|Test Sequence]] can establish the order, conditional progression, repetition, and termination of the Sequenced Test Objects performed during Test Execution.
 +
 +A [[public:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] identifies and records a particular occurrence of Test Execution.
 +
 +===== Definition =====
 +
 +//performance of one or more [[public:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Objects]] within a [[public:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] to produce one or more [[public:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]]//
 +
 +===== Source =====
 +
 +Adapted from:
 +
 +  * [[https://www.omg.org/spec/TestIF/1.0/Beta3/PDF|OMG Test Information Interchange Format (TestIF), Version 1.0 Beta 3, Sections 1.3, 7.3.2, 7.3.9, 7.3.10, and 7.3.11]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Object]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:t:test_sequence|Test Sequence]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]]
 +  * [[public:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]]
 +  * DIDO Reference Implementation Conceptual Model
 +  * DIDO-TE draft Requirements Register
 +
 +OMG TestIF models the information used to define tests and the TestRuns and TestResults produced when those tests execute.
 +
 +OMG TestIF does not define Test Execution as a separate TestIF model element. This definition introduces Test Execution as the activity that occurs between the applicable test definitions and the resulting Test Run and Test Results.
 +
 +===== Note =====
 +
 +Test Execution differs from a [[public:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]]:
 +
 +  * Test Execution is the activity of performing the applicable Sequenced Test Objects
 +  * A Test Run identifies and records a particular occurrence of that activity
 +
 +Test Execution differs from a [[public:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]]:
 +
 +  * Test Execution performs the applicable test activities
 +  * A Test Result records an outcome produced by those activities
 +
 +Test Execution differs from a [[public:99_annexes:annex-b-terms-and-definitions:t:test_sequence|Test Sequence]]:
 +
 +  * A Test Sequence specifies the order and progression of Sequenced Test Objects
 +  * Test Execution performs those objects according to the applicable Test Sequence
 +
 +Test Execution differs from a [[public:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]]:
 +
 +  * A Test Environment identifies the conditions, resources, configurations, and dependencies required to conduct testing
 +  * Test Execution is the activity conducted within those conditions and using those resources
 +
 +Test Execution differs from a [[public:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executable]]:
 +
 +  * A Test Executable contains or references [[public:99_annexes:annex-b-terms-and-definitions:e:executable_instruction|Executable Instructions]]
 +  * Test Execution causes the applicable Execution Facilities to interpret or perform those instructions
 +
 +Test Execution can use automated, manual, or combined procedures.
 +
 +Manual Test Execution can produce valid Test Results when the applicable test definition identifies:
 +
 +  * The required actions
 +  * The responsible participant
 +  * The required observations
 +  * The applicable Expected Results
 +  * The applicable Verdict criteria
 +  * The required Evidence
 +
 +Test Execution can perform Sequenced Test Objects:
 +
 +  * Sequentially
 +  * Concurrently
 +  * Conditionally
 +  * Iteratively
 +  * In response to an event
 +  * According to another progression established by the applicable Test Sequence
 +
 +Starting Test Execution does not establish successful completion. Test Execution can:
 +
 +  * Complete successfully
 +  * Complete with one or more failed Test Results
 +  * Terminate because of an error
 +  * Terminate because of a timeout
 +  * Stop because a precondition is not satisfied
 +  * Stop because a required resource is unavailable
 +  * Be interrupted
 +  * Be canceled
 +  * Remain incomplete
 +
 +The applicable [[public:99_annexes:annex-b-terms-and-definitions:a:acceptance_criteria|Acceptance Criteria]] or [[public:99_annexes:annex-b-terms-and-definitions:g:governance_policy|Governance Policy]] determines how incomplete, interrupted, canceled, repeated, or failed Test Execution affects the resulting Verdict or [[public:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]].
 +
 +Repeated Test Execution does not necessarily reproduce the same Test Results. Reproducibility depends on preserving or controlling the applicable:
 +
 +  * Sequenced Test Object Versions
 +  * Test Item Versions and states
 +  * Test Environment
 +  * Test Argument Values
 +  * Executable Artifact Versions and content digests
 +  * Execution Facility Versions and configurations
 +  * Dependencies
 +  * Starting conditions
 +  * Timing conditions
 +  * External interactions
 +  * Governance Policies
 +
 +A reproducible Test Run should preserve sufficient Evidence, Provenance, and Traceability to identify these conditions.
 +
 +===== Example =====
 +
 +A [[public:99_annexes:annex-b-terms-and-definitions:t:test_case|Test Case]] evaluates whether a selected [[public:99_annexes:annex-b-terms-and-definitions:n:node_set|Node Set]] reaches the expected distributed state after submission of a proposed transaction.
 +
 +Test Execution:
 +
 +  * Establishes the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]]
 +  * Confirms the required Nodes and Execution Facilities are available
 +  * Loads the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:executable_artifact|Executable Artifacts]]
 +  * Supplies the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_argument_value|Test Argument Values]]
 +  * Performs the applicable [[private:99_annexes:annex-b-terms-and-definitions:t:test_steps|Test Steps]]
 +  * Submits the proposed transaction through the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:endpoint|Endpoint]]
 +  * Observes the resulting state of each applicable [[public:99_annexes:annex-b-terms-and-definitions:n:node|Node]]
 +  * Compares the observed states with the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:expected_result|Expected Results]]
 +  * Produces the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]]
 +  * Preserves the supporting [[public:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]]
 +
 +The [[public:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] records this occurrence of Test Execution and maintains Traceability among:
 +
 +  * The Test Case
 +  * The performed Test Steps
 +  * The applicable Test Items
 +  * The applicable Node Set
 +  * The Test Environment
 +  * The Test Argument Values
 +  * The Executable Artifacts
 +  * The Execution Facilities
 +  * The Test Results
 +  * The Evidence
 +  * The resulting [[public:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]]
 +
 +----
 +
 +<WRAP centeralign>
 +© 2026 Dido Solutions, Inc. and Jackrabbit Consulting, Inc.
 +</WRAP>