| |
| dido:99_annexes:annex-b-terms-and-definitions:e:expected_result [2026/08/04 13:37] – created nick_dido | dido:99_annexes:annex-b-terms-and-definitions:e:expected_result [2026/08/07 01:07] (current) – nick_dido |
|---|
| ===== Discussion ===== | ===== Discussion ===== |
| |
| An Expected Result specifies an anticipated value, state, event, condition, behavior, or outcome associated with the performance of a [[dido:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Object]] or [[dido:99_annexes:annex-b-terms-and-definitions:s:sequence_step|Sequence Step]]. | An Expected Result specifies an anticipated value, state, state transition, event, condition, behavior, response, side effect, or outcome associated with the performance of a [[dido:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Object]] or [[dido:99_annexes:annex-b-terms-and-definitions:s:sequence_step|Sequence Step]]. |
| |
| An Expected Result provides a basis for evaluating an observed or derived [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]]. | An Expected Result provides the basis for evaluating an observed or derived [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]]. The Expected Result states what the test anticipates. The Test Result records what occurred. |
| |
| An Expected Result can specify: | An Expected Result expresses one or more objectively evaluable conditions, including: |
| |
| * An anticipated value | * An anticipated value or permitted set of values |
| * An anticipated state | * An anticipated state or state transition |
| * An anticipated state transition | * An anticipated event, response, or behavior |
| * An anticipated event | |
| * An anticipated response | |
| * An anticipated behavior | |
| * An anticipated side effect | * An anticipated side effect |
| * An anticipated absence of an event or behavior | * The absence of an event, response, behavior, or side effect |
| * A required range | * A required range, threshold, tolerance, or precision |
| * A required threshold | * A required ordering or temporal relationship |
| * A required ordering | * A required completion or error condition |
| * A required timing condition | * A permitted or prohibited outcome |
| * A required completion condition | |
| * A required error condition | |
| * A permitted set of outcomes | |
| * A prohibited outcome | |
| |
| An Expected Result can identify: | An Expected Result applies to a: |
| | |
| * Its identity | |
| * Its name | |
| * Its description | |
| * Its expected value or condition | |
| * Its value type | |
| * Its unit of measure | |
| * Its permitted range | |
| * Its tolerance | |
| * Its required precision | |
| * Its applicable time limit | |
| * Its applicable [[dido:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Object]] | |
| * Its applicable [[dido:99_annexes:annex-b-terms-and-definitions:s:sequence_step|Sequence Step]] | |
| * Its applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_item|Test Items]] | |
| * Its applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_argument_value|Test Argument Values]] | |
| * Its comparison method | |
| * Its evaluation criteria | |
| * Its applicable [[dido:99_annexes:annex-b-terms-and-definitions:a:acceptance_criteria|Acceptance Criteria]] | |
| * Its applicable [[dido:99_annexes:annex-b-terms-and-definitions:g:governance_policy|Governance Policies]] | |
| * Its [[dido:99_annexes:annex-b-terms-and-definitions:p:provenance|Provenance]] | |
| * Its [[dido:99_annexes:annex-b-terms-and-definitions:t:traceability|Traceability]] | |
| | |
| An Expected Result can apply to a: | |
| |
| * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_set|Test Set]] | * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_set|Test Set]] |
| * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_step|Test Step]] | * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_step|Test Step]] |
| * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executable]] | * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executable]] |
| * Sequence Step | * [[dido:99_annexes:annex-b-terms-and-definitions:s:sequence_step|Sequence Step]] |
| |
| During or after [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution|Test Execution]], an evaluator or automated mechanism can compare the applicable Test Result with the Expected Result and assign a [[dido:99_annexes:annex-b-terms-and-definitions:v:verdict|Verdict]] according to defined evaluation criteria. | During or after [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution|Test Execution]], an evaluator or automated mechanism compares the applicable Test Result with the Expected Result. The evaluator or mechanism then applies the governing [[dido:99_annexes:annex-b-terms-and-definitions:e:evaluation_criterion|Evaluation Criteria]] to assign a [[dido:99_annexes:annex-b-terms-and-definitions:v:verdict|Verdict]]. |
| |
| ===== Definition ===== | ===== Definition ===== |
| |
| //anticipated value, state, event, condition, behavior, or outcome used to evaluate a [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]]// | //anticipated value, state, state transition, event, condition, behavior, response, side effect, or outcome used as a basis for evaluating a [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]]// |
| |
| ===== Source ===== | ===== Source ===== |
| |
| * [[https://www.omg.org/spec/TestIF/1.0/Beta3/PDF|OMG Test Information Interchange Format (TestIF), Version 1.0 Beta 3, Sections 7.2 and 7.4, expectedResult]] | * [[https://www.omg.org/spec/TestIF/1.0/Beta3/PDF|OMG Test Information Interchange Format (TestIF), Version 1.0 Beta 3, Sections 7.2 and 7.4, expectedResult]] |
| * [[dido:99_annexes:annex-b-terms-and-definitions:s:sequenced_test_object|Sequenced Test Object]] | |
| * [[dido:99_annexes:annex-b-terms-and-definitions:s:sequence_step|Sequence Step]] | |
| * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]] | |
| * [[dido:99_annexes:annex-b-terms-and-definitions:v:verdict|Verdict]] | |
| * DIDO Reference Implementation Conceptual Model | * DIDO Reference Implementation Conceptual Model |
| * DIDO-TE draft Requirements Register | * DIDO-TE draft Requirements Register |
| |
| OMG TestIF provides the predefined ''expectedResult'' attribute for representing expected-result values associated with a SequencedTestObject or SequenceStep. | OMG TestIF defines the ''expectedResult'' attribute for representing expected-result values associated with a ''SequencedTestObject'' or ''SequenceStep''. |
| |
| OMG TestIF permits an Expected Result to contain one or more values when multiple values are necessary to express the anticipated outcome. | OMG TestIF permits the ''expectedResult'' attribute to contain multiple values when the anticipated result requires more than one value. |
| |
| This definition generalizes Expected Result beyond a single value while preserving its TestIF role as the anticipated outcome used to evaluate a Test Result. | This definition generalizes the TestIF concept beyond a single value while preserving its role as the anticipated result used to evaluate a Test Result. |
| |
| ===== Note ===== | ===== Note ===== |
| |
| Use the singular glossary term Expected Result. Link plural usage to the same controlling page: | Use **Expected Result** as the singular glossary term. Link plural occurrences to this page: |
| |
| <code dokuwiki> | <code dokuwiki> |
| An Expected Result differs from a [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]]: | An Expected Result differs from a [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]]: |
| |
| * An Expected Result specifies an anticipated outcome | * An Expected Result specifies an anticipated result |
| * A Test Result records an observed or derived outcome | * A Test Result records an observed or derived result |
| |
| An Expected Result differs from a [[dido:99_annexes:annex-b-terms-and-definitions:v:verdict|Verdict]]: | An Expected Result differs from a [[dido:99_annexes:annex-b-terms-and-definitions:v:verdict|Verdict]]: |
| |
| * An Expected Result identifies what is anticipated or required | * An Expected Result states what the test anticipates |
| * A Verdict classifies a Test Result according to defined evaluation criteria | * A Verdict classifies a Test Result according to the governing Evaluation Criteria |
| |
| An Expected Result differs from [[dido:99_annexes:annex-b-terms-and-definitions:a:acceptance_criteria|Acceptance Criteria]]: | An Expected Result differs from [[dido:99_annexes:annex-b-terms-and-definitions:a:acceptance_criteria|Acceptance Criteria]]: |
| |
| * An Expected Result specifies an anticipated outcome for a Sequenced Test Object or Sequence Step | * An Expected Result specifies an anticipated result associated with a test |
| * Acceptance Criteria establish the conditions used to determine whether an evaluated subject or outcome is acceptable | * Acceptance Criteria establish the conditions governing acceptance of an evaluated subject or outcome |
| |
| Acceptance Criteria can reference one or more Expected Results. | Acceptance Criteria reference applicable Expected Results when acceptance depends upon the results of testing. |
| |
| An Expected Result does not necessarily require exact equality. The Expected Result can establish: | An Expected Result identifies or references the information required for repeatable evaluation, including: |
| |
| * A range | * The applicable Sequenced Test Object or Sequence Step |
| * A minimum | * The applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_item|Test Item]] |
| * A maximum | * The applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_argument_value|Test Argument Values]] |
| * A tolerance | * The anticipated value, state, event, condition, behavior, response, side effect, or outcome |
| * A pattern | * Any applicable unit of measure |
| * A set of permitted values | * Any applicable range, threshold, tolerance, or precision |
| * A temporal relationship | * Any applicable ordering or timing condition |
| * A required state transition | * The comparison method |
| * The occurrence of an event | * The governing Evaluation Criteria |
| * The absence of an event | * The applicable Acceptance Criteria |
| * Another objectively evaluable condition | * The applicable [[dido:99_annexes:annex-b-terms-and-definitions:g:governance_policy|Governance Policies]] |
| | * Its [[dido:99_annexes:annex-b-terms-and-definitions:p:provenance|Provenance]] |
| | * Its [[dido:99_annexes:annex-b-terms-and-definitions:t:traceability|Traceability]] |
| |
| An Expected Result must provide sufficient information to support repeatable evaluation. Phrases such as “works correctly,” “responds quickly,” “performs adequately,” or “produces an appropriate result” do not establish objectively evaluable Expected Results without additional criteria. | An Expected Result containing multiple related values preserves the relationship among those values. |
| |
| A Sequenced Test Object can define a reusable Expected Result. A Sequence Step can refine or supply an occurrence-specific Expected Result when the expected outcome depends on the Sequence Step’s: | A Sequenced Test Object defines a reusable Expected Result. A Sequence Step refines or supplies an occurrence-specific Expected Result when the anticipated result depends upon: |
| |
| * Test Argument Values | * Test Argument Values |
| * Position | * The position of the Sequence Step |
| * Preceding state | * A preceding state |
| * Execution path | * The execution path |
| * Test Item | * The applicable Test Item |
| * Test Environment | * The [[dido:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] |
| * Other occurrence-specific conditions | * Other occurrence-specific conditions |
| |
| An Expected Result can contain multiple related expected values when the evaluated outcome has multiple characteristics. The definition must preserve the relationship among those values. | An Expected Result provides sufficient information for objective and repeatable evaluation. Expressions such as “works correctly,” “responds quickly,” “performs adequately,” or “produces an appropriate result” do not establish an objectively evaluable Expected Result without additional criteria. |
| |
| Failure to observe an Expected Result does not automatically establish the cause of the difference. The Test Result and supporting [[dido:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]] must preserve the observed outcome. | A difference between an Expected Result and a Test Result does not establish the cause of the difference. The Test Result and supporting [[dido:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]] preserve the observed or derived result. |
| |
| An Expected Result does not by itself establish a [[dido:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]]. The Validation Decision applies the governing decision criteria to the relevant Test Results, Verdicts, and Evidence. | An Expected Result does not establish a [[dido:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]]. The Validation Decision applies the governing decision criteria to the applicable Test Results, Verdicts, and Evidence. |
| |
| ===== Example ===== | ===== Example ===== |
| |
| A [[dido:99_annexes:annex-b-terms-and-definitions:t:test_step|Test Step]] requires a selected [[dido:99_annexes:annex-b-terms-and-definitions:n:node|Node]] to process a valid transaction. | A [[dido:99_annexes:annex-b-terms-and-definitions:t:test_step|Test Step]] directs a selected [[dido:99_annexes:annex-b-terms-and-definitions:n:node|Node]] to process a valid transaction. |
| |
| The Expected Result specifies that: | The Expected Result specifies that: |
| |
| * The Node accepts the transaction | * The Node accepts the transaction |
| * The Node returns a transaction identifier | * The Node returns a nonempty transaction identifier |
| * The transaction identifier is not empty | |
| * The Node records the committed transaction state | * The Node records the committed transaction state |
| * The committed state references the preceding state | * The committed state references the preceding state |
| * The Node produces no duplicate committed state | * The Node produces no duplicate committed state |
| |
| During [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution|Test Execution]], the applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]] records: | During Test Execution, the applicable Test Result records: |
| |
| * The observed response | * The observed response |
| * The predecessor-state reference | * The predecessor-state reference |
| * The measured response time | * The measured response time |
| * Any duplicate state | * Any duplicate committed state |
| * The supporting Evidence | * The supporting Evidence |
| |
| The evaluator compares the Test Result with the Expected Result and assigns the applicable Verdict according to the defined evaluation criteria. | The evaluator compares the Test Result with the Expected Result and applies the governing Evaluation Criteria to assign the applicable Verdict. |
| |
| ---- | ---- |