| Both sides previous revision Previous revision | |
| public:99_annexes:annex-b-terms-and-definitions:t:test_execution [2026/09/12 17:42] – ↷ Page moved from private:99_annexes:annex-b-terms-and-definitions:t:test_execution to public:99_annexes:annex-b-terms-and-definitions:t:test_execution nick_dido | public:99_annexes:annex-b-terms-and-definitions:t:test_execution [2026/09/12 17:43] (current) – ↷ Links adapted because of a move operation nick_dido |
|---|
| ====== Test Execution ====== | ====== Test Execution ====== |
| |
| [[private:99_annexes:annex-b-terms-and-definitions:start|Go up to Terms and Definitions]] | [[public:99_annexes:annex-b-terms-and-definitions:start|Go up to Terms and Definitions]] |
| |
| ===== Discussion ===== | ===== Discussion ===== |
| |
| Test Execution is the performance of one or more [[private:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Objects]] within a [[private:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] to produce one or more [[private:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]]. | Test Execution is the performance of one or more [[public:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Objects]] within a [[public:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] to produce one or more [[public:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]]. |
| |
| Test Execution can perform: | Test Execution can perform: |
| |
| * A [[private:99_annexes:annex-b-terms-and-definitions:t:test_set|Test Set]] | * A [[public:99_annexes:annex-b-terms-and-definitions:t:test_set|Test Set]] |
| * A [[private:99_annexes:annex-b-terms-and-definitions:t:test_case|Test Case]] | * A [[public:99_annexes:annex-b-terms-and-definitions:t:test_case|Test Case]] |
| * A [[private:99_annexes:annex-b-terms-and-definitions:t:test_step|Test Step]] | * A [[public:99_annexes:annex-b-terms-and-definitions:t:test_step|Test Step]] |
| * A [[private:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executable]] | * A [[public:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executable]] |
| |
| Test Execution can include: | Test Execution can include: |
| * Establishing the applicable Test Environment | * Establishing the applicable Test Environment |
| * Confirming applicable preconditions | * Confirming applicable preconditions |
| * Selecting the applicable [[private:99_annexes:annex-b-terms-and-definitions:t:test_item|Test Items]] | * Selecting the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_item|Test Items]] |
| * Selecting the applicable [[private:99_annexes:annex-b-terms-and-definitions:n:node_set|Node Sets]] | * Selecting the applicable [[public:99_annexes:annex-b-terms-and-definitions:n:node_set|Node Sets]] |
| * Selecting the applicable [[private:99_annexes:annex-b-terms-and-definitions:d:deployment_target|Deployment Targets]] | * Selecting the applicable [[public:99_annexes:annex-b-terms-and-definitions:d:deployment_target|Deployment Targets]] |
| * Resolving the applicable [[private:99_annexes:annex-b-terms-and-definitions:t:test_argument|Test Arguments]] | * Resolving the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_argument|Test Arguments]] |
| * Supplying the applicable [[private:99_annexes:annex-b-terms-and-definitions:t:test_argument_value|Test Argument Values]] | * Supplying the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_argument_value|Test Argument Values]] |
| * Selecting the applicable [[private:99_annexes:annex-b-terms-and-definitions:e:execution_facility|Execution Facilities]] | * Selecting the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:execution_facility|Execution Facilities]] |
| * Retrieving the applicable [[private:99_annexes:annex-b-terms-and-definitions:e:executable_artifact|Executable Artifacts]] | * Retrieving the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:executable_artifact|Executable Artifacts]] |
| * Interpreting or performing the applicable [[private:99_annexes:annex-b-terms-and-definitions:e:executable_instruction|Executable Instructions]] | * Interpreting or performing the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:executable_instruction|Executable Instructions]] |
| * Observing the behavior of the Test Items | * Observing the behavior of the Test Items |
| * Measuring applicable conditions and outcomes | * Measuring applicable conditions and outcomes |
| * Comparing observed outcomes with [[private:99_annexes:annex-b-terms-and-definitions:e:expected_result|Expected Results]] | * Comparing observed outcomes with [[public:99_annexes:annex-b-terms-and-definitions:e:expected_result|Expected Results]] |
| * Assigning applicable [[private:99_annexes:annex-b-terms-and-definitions:v:verdict|Verdicts]] | * Assigning applicable [[public:99_annexes:annex-b-terms-and-definitions:v:verdict|Verdicts]] |
| * Producing Test Results | * Producing Test Results |
| * Collecting [[private:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]] | * Collecting [[public:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]] |
| * Preserving [[private:99_annexes:annex-b-terms-and-definitions:p:provenance|Provenance]] | * Preserving [[public:99_annexes:annex-b-terms-and-definitions:p:provenance|Provenance]] |
| * Maintaining [[private:99_annexes:annex-b-terms-and-definitions:t:traceability|Traceability]] | * Maintaining [[public:99_annexes:annex-b-terms-and-definitions:t:traceability|Traceability]] |
| |
| A [[private:99_annexes:annex-b-terms-and-definitions:t:test_sequence|Test Sequence]] can establish the order, conditional progression, repetition, and termination of the Sequenced Test Objects performed during Test Execution. | A [[public:99_annexes:annex-b-terms-and-definitions:t:test_sequence|Test Sequence]] can establish the order, conditional progression, repetition, and termination of the Sequenced Test Objects performed during Test Execution. |
| |
| A [[private:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] identifies and records a particular occurrence of Test Execution. | A [[public:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] identifies and records a particular occurrence of Test Execution. |
| |
| ===== Definition ===== | ===== Definition ===== |
| |
| //performance of one or more [[private:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Objects]] within a [[private:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] to produce one or more [[private:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]]// | //performance of one or more [[public:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Objects]] within a [[public:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] to produce one or more [[public:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]]// |
| |
| ===== Source ===== | ===== Source ===== |
| |
| * [[https://www.omg.org/spec/TestIF/1.0/Beta3/PDF|OMG Test Information Interchange Format (TestIF), Version 1.0 Beta 3, Sections 1.3, 7.3.2, 7.3.9, 7.3.10, and 7.3.11]] | * [[https://www.omg.org/spec/TestIF/1.0/Beta3/PDF|OMG Test Information Interchange Format (TestIF), Version 1.0 Beta 3, Sections 1.3, 7.3.2, 7.3.9, 7.3.10, and 7.3.11]] |
| * [[private:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Object]] | * [[public:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Object]] |
| * [[private:99_annexes:annex-b-terms-and-definitions:t:test_sequence|Test Sequence]] | * [[public:99_annexes:annex-b-terms-and-definitions:t:test_sequence|Test Sequence]] |
| * [[private:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] | * [[public:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] |
| * [[private:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]] | * [[public:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]] |
| * [[private:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] | * [[public:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] |
| * DIDO Reference Implementation Conceptual Model | * DIDO Reference Implementation Conceptual Model |
| * DIDO-TE draft Requirements Register | * DIDO-TE draft Requirements Register |
| ===== Note ===== | ===== Note ===== |
| |
| Test Execution differs from a [[private:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]]: | Test Execution differs from a [[public:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]]: |
| |
| * Test Execution is the activity of performing the applicable Sequenced Test Objects | * Test Execution is the activity of performing the applicable Sequenced Test Objects |
| * A Test Run identifies and records a particular occurrence of that activity | * A Test Run identifies and records a particular occurrence of that activity |
| |
| Test Execution differs from a [[private:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]]: | Test Execution differs from a [[public:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]]: |
| |
| * Test Execution performs the applicable test activities | * Test Execution performs the applicable test activities |
| * A Test Result records an outcome produced by those activities | * A Test Result records an outcome produced by those activities |
| |
| Test Execution differs from a [[private:99_annexes:annex-b-terms-and-definitions:t:test_sequence|Test Sequence]]: | Test Execution differs from a [[public:99_annexes:annex-b-terms-and-definitions:t:test_sequence|Test Sequence]]: |
| |
| * A Test Sequence specifies the order and progression of Sequenced Test Objects | * A Test Sequence specifies the order and progression of Sequenced Test Objects |
| * Test Execution performs those objects according to the applicable Test Sequence | * Test Execution performs those objects according to the applicable Test Sequence |
| |
| Test Execution differs from a [[private:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]]: | Test Execution differs from a [[public:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]]: |
| |
| * A Test Environment identifies the conditions, resources, configurations, and dependencies required to conduct testing | * A Test Environment identifies the conditions, resources, configurations, and dependencies required to conduct testing |
| * Test Execution is the activity conducted within those conditions and using those resources | * Test Execution is the activity conducted within those conditions and using those resources |
| |
| Test Execution differs from a [[private:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executable]]: | Test Execution differs from a [[public:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executable]]: |
| |
| * A Test Executable contains or references [[private:99_annexes:annex-b-terms-and-definitions:e:executable_instruction|Executable Instructions]] | * A Test Executable contains or references [[public:99_annexes:annex-b-terms-and-definitions:e:executable_instruction|Executable Instructions]] |
| * Test Execution causes the applicable Execution Facilities to interpret or perform those instructions | * Test Execution causes the applicable Execution Facilities to interpret or perform those instructions |
| |
| * Remain incomplete | * Remain incomplete |
| |
| The applicable [[private:99_annexes:annex-b-terms-and-definitions:a:acceptance_criteria|Acceptance Criteria]] or [[private:99_annexes:annex-b-terms-and-definitions:g:governance_policy|Governance Policy]] determines how incomplete, interrupted, canceled, repeated, or failed Test Execution affects the resulting Verdict or [[private:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]]. | The applicable [[public:99_annexes:annex-b-terms-and-definitions:a:acceptance_criteria|Acceptance Criteria]] or [[public:99_annexes:annex-b-terms-and-definitions:g:governance_policy|Governance Policy]] determines how incomplete, interrupted, canceled, repeated, or failed Test Execution affects the resulting Verdict or [[public:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]]. |
| |
| Repeated Test Execution does not necessarily reproduce the same Test Results. Reproducibility depends on preserving or controlling the applicable: | Repeated Test Execution does not necessarily reproduce the same Test Results. Reproducibility depends on preserving or controlling the applicable: |
| ===== Example ===== | ===== Example ===== |
| |
| A [[private:99_annexes:annex-b-terms-and-definitions:t:test_case|Test Case]] evaluates whether a selected [[private:99_annexes:annex-b-terms-and-definitions:n:node_set|Node Set]] reaches the expected distributed state after submission of a proposed transaction. | A [[public:99_annexes:annex-b-terms-and-definitions:t:test_case|Test Case]] evaluates whether a selected [[public:99_annexes:annex-b-terms-and-definitions:n:node_set|Node Set]] reaches the expected distributed state after submission of a proposed transaction. |
| |
| Test Execution: | Test Execution: |
| |
| * Establishes the applicable [[private:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] | * Establishes the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] |
| * Confirms the required Nodes and Execution Facilities are available | * Confirms the required Nodes and Execution Facilities are available |
| * Loads the applicable [[private:99_annexes:annex-b-terms-and-definitions:e:executable_artifact|Executable Artifacts]] | * Loads the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:executable_artifact|Executable Artifacts]] |
| * Supplies the applicable [[private:99_annexes:annex-b-terms-and-definitions:t:test_argument_value|Test Argument Values]] | * Supplies the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_argument_value|Test Argument Values]] |
| * Performs the applicable [[private:99_annexes:annex-b-terms-and-definitions:t:test_steps|Test Steps]] | * Performs the applicable [[private:99_annexes:annex-b-terms-and-definitions:t:test_steps|Test Steps]] |
| * Submits the proposed transaction through the applicable [[private:99_annexes:annex-b-terms-and-definitions:e:endpoint|Endpoint]] | * Submits the proposed transaction through the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:endpoint|Endpoint]] |
| * Observes the resulting state of each applicable [[private:99_annexes:annex-b-terms-and-definitions:n:node|Node]] | * Observes the resulting state of each applicable [[public:99_annexes:annex-b-terms-and-definitions:n:node|Node]] |
| * Compares the observed states with the applicable [[private:99_annexes:annex-b-terms-and-definitions:e:expected_result|Expected Results]] | * Compares the observed states with the applicable [[public:99_annexes:annex-b-terms-and-definitions:e:expected_result|Expected Results]] |
| * Produces the applicable [[private:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]] | * Produces the applicable [[public:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]] |
| * Preserves the supporting [[private:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]] | * Preserves the supporting [[public:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]] |
| |
| The [[private:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] records this occurrence of Test Execution and maintains Traceability among: | The [[public:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Run]] records this occurrence of Test Execution and maintains Traceability among: |
| |
| * The Test Case | * The Test Case |
| * The Test Results | * The Test Results |
| * The Evidence | * The Evidence |
| * The resulting [[private:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]] | * The resulting [[public:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]] |
| |
| ---- | ---- |