Expected Result

Go up to Terms and Definitions

An Expected Result specifies an anticipated value, state, state transition, event, condition, behavior, response, side effect, or outcome associated with the performance of a Sequenced Test Object or Sequence Step.

An Expected Result provides the basis for evaluating an observed or derived Test Result. The Expected Result states what the test anticipates. The Test Result records what occurred.

An Expected Result expresses one or more objectively evaluable conditions, including:

• An anticipated value or permitted set of values
• An anticipated state or state transition
• An anticipated event, response, or behavior
• An anticipated side effect
• The absence of an event, response, behavior, or side effect
• A required range, threshold, tolerance, or precision
• A required ordering or temporal relationship
• A required completion or error condition
• A permitted or prohibited outcome

An Expected Result applies to a:

• Test Set
• Test Case
• Test Step
• Test Executable
• Sequence Step

During or after Test Execution, an evaluator or automated mechanism compares the applicable Test Result with the Expected Result. The evaluator or mechanism then applies the governing Evaluation Criteria to assign a Verdict.

anticipated value, state, state transition, event, condition, behavior, response, side effect, or outcome used as a basis for evaluating a Test Result

Adapted from:

• OMG Test Information Interchange Format (TestIF), Version 1.0 Beta 3, Sections 7.2 and 7.4, expectedResult
• DIDO Reference Implementation Conceptual Model
• DIDO-TE draft Requirements Register

OMG TestIF defines the expectedResult attribute for representing expected-result values associated with a SequencedTestObject or SequenceStep.

OMG TestIF permits the expectedResult attribute to contain multiple values when the anticipated result requires more than one value.

This definition generalizes the TestIF concept beyond a single value while preserving its role as the anticipated result used to evaluate a Test Result.

Use Expected Result as the singular glossary term. Link plural occurrences to this page:

[[dido:99_annexes:annex-b-terms-and-definitions:e:expected_result|Expected Results]]

An Expected Result differs from a Test Result:

• An Expected Result specifies an anticipated result
• A Test Result records an observed or derived result

An Expected Result differs from a Verdict:

• An Expected Result states what the test anticipates
• A Verdict classifies a Test Result according to the governing Evaluation Criteria

An Expected Result differs from Acceptance Criteria:

• An Expected Result specifies an anticipated result associated with a test
• Acceptance Criteria establish the conditions governing acceptance of an evaluated subject or outcome

Acceptance Criteria reference applicable Expected Results when acceptance depends upon the results of testing.

An Expected Result identifies or references the information required for repeatable evaluation, including:

• The applicable Sequenced Test Object or Sequence Step
• The applicable Test Item
• The applicable Test Argument Values
• The anticipated value, state, event, condition, behavior, response, side effect, or outcome
• Any applicable unit of measure
• Any applicable range, threshold, tolerance, or precision
• Any applicable ordering or timing condition
• The comparison method
• The governing Evaluation Criteria
• The applicable Acceptance Criteria
• The applicable Governance Policies
• Its Provenance
• Its Traceability

An Expected Result containing multiple related values preserves the relationship among those values.

A Sequenced Test Object defines a reusable Expected Result. A Sequence Step refines or supplies an occurrence-specific Expected Result when the anticipated result depends upon:

• Test Argument Values
• The position of the Sequence Step
• A preceding state
• The execution path
• The applicable Test Item
• The Test Environment
• Other occurrence-specific conditions

An Expected Result provides sufficient information for objective and repeatable evaluation. Expressions such as “works correctly,” “responds quickly,” “performs adequately,” or “produces an appropriate result” do not establish an objectively evaluable Expected Result without additional criteria.

A difference between an Expected Result and a Test Result does not establish the cause of the difference. The Test Result and supporting Evidence preserve the observed or derived result.

An Expected Result does not establish a Validation Decision. The Validation Decision applies the governing decision criteria to the applicable Test Results, Verdicts, and Evidence.

A Test Step directs a selected Node to process a valid transaction.

The Expected Result specifies that:

• The Node accepts the transaction
• The Node returns a nonempty transaction identifier
• The Node records the committed transaction state
• The committed state references the preceding state
• The response occurs within five seconds
• The Node produces no duplicate committed state

During Test Execution, the applicable Test Result records:

• The observed response
• The returned transaction identifier
• The resulting transaction state
• The predecessor-state reference
• The measured response time
• Any duplicate committed state
• The supporting Evidence

The evaluator compares the Test Result with the Expected Result and applies the governing Evaluation Criteria to assign the applicable Verdict.