Expected Result

Go up to Terms and Definitions

An Expected Result specifies an anticipated value, state, event, condition, behavior, or outcome associated with the performance of a Sequenced Test Object or Sequence Step.

An Expected Result provides a basis for evaluating an observed or derived Test Result.

An Expected Result can specify:

• An anticipated value
• An anticipated state
• An anticipated state transition
• An anticipated event
• An anticipated response
• An anticipated behavior
• An anticipated side effect
• An anticipated absence of an event or behavior
• A required range
• A required threshold
• A required ordering
• A required timing condition
• A required completion condition
• A required error condition
• A permitted set of outcomes
• A prohibited outcome

An Expected Result can identify:

• Its identity
• Its name
• Its description
• Its expected value or condition
• Its value type
• Its unit of measure
• Its permitted range
• Its tolerance
• Its required precision
• Its applicable time limit
• Its applicable Sequenced Test Object
• Its applicable Sequence Step
• Its applicable Test Items
• Its applicable Test Argument Values
• Its comparison method
• Its evaluation criteria
• Its applicable Acceptance Criteria
• Its applicable Governance Policies
• Its Provenance
• Its Traceability

An Expected Result can apply to a:

• Test Set
• Test Case
• Test Step
• Test Executable
• Sequence Step

During or after Test Execution, an evaluator or automated mechanism can compare the applicable Test Result with the Expected Result and assign a Verdict according to defined evaluation criteria.

anticipated value, state, event, condition, behavior, or outcome used to evaluate a Test Result

Adapted from:

• OMG Test Information Interchange Format (TestIF), Version 1.0 Beta 3, Sections 7.2 and 7.4, expectedResult
• Sequenced Test Object
• Sequence Step
• Test Result
• Verdict
• DIDO Reference Implementation Conceptual Model
• DIDO-TE draft Requirements Register

OMG TestIF provides the predefined expectedResult attribute for representing expected-result values associated with a SequencedTestObject or SequenceStep.

OMG TestIF permits an Expected Result to contain one or more values when multiple values are necessary to express the anticipated outcome.

This definition generalizes Expected Result beyond a single value while preserving its TestIF role as the anticipated outcome used to evaluate a Test Result.

Use the singular glossary term Expected Result. Link plural usage to the same controlling page:

[[dido:99_annexes:annex-b-terms-and-definitions:e:expected_result|Expected Results]]

An Expected Result differs from a Test Result:

• An Expected Result specifies an anticipated outcome
• A Test Result records an observed or derived outcome

An Expected Result differs from a Verdict:

• An Expected Result identifies what is anticipated or required
• A Verdict classifies a Test Result according to defined evaluation criteria

An Expected Result differs from Acceptance Criteria:

• An Expected Result specifies an anticipated outcome for a Sequenced Test Object or Sequence Step
• Acceptance Criteria establish the conditions used to determine whether an evaluated subject or outcome is acceptable

Acceptance Criteria can reference one or more Expected Results.

An Expected Result does not necessarily require exact equality. The Expected Result can establish:

• A range
• A minimum
• A maximum
• A tolerance
• A pattern
• A set of permitted values
• A temporal relationship
• A required state transition
• The occurrence of an event
• The absence of an event
• Another objectively evaluable condition

An Expected Result must provide sufficient information to support repeatable evaluation. Phrases such as “works correctly,” “responds quickly,” “performs adequately,” or “produces an appropriate result” do not establish objectively evaluable Expected Results without additional criteria.

A Sequenced Test Object can define a reusable Expected Result. A Sequence Step can refine or supply an occurrence-specific Expected Result when the expected outcome depends on the Sequence Step’s:

• Test Argument Values
• Position
• Preceding state
• Execution path
• Test Item
• Test Environment
• Other occurrence-specific conditions

An Expected Result can contain multiple related expected values when the evaluated outcome has multiple characteristics. The definition must preserve the relationship among those values.

Failure to observe an Expected Result does not automatically establish the cause of the difference. The Test Result and supporting Evidence must preserve the observed outcome.

An Expected Result does not by itself establish a Validation Decision. The Validation Decision applies the governing decision criteria to the relevant Test Results, Verdicts, and Evidence.

A Test Step requires a selected Node to process a valid transaction.

The Expected Result specifies that:

• The Node accepts the transaction
• The Node returns a transaction identifier
• The transaction identifier is not empty
• The Node records the committed transaction state
• The committed state references the preceding state
• The response occurs within five seconds
• The Node produces no duplicate committed state

During Test Execution, the applicable Test Result records:

• The observed response
• The returned transaction identifier
• The resulting transaction state
• The predecessor-state reference
• The measured response time
• Any duplicate state
• The supporting Evidence

The evaluator compares the Test Result with the Expected Result and assigns the applicable Verdict according to the defined evaluation criteria.