Evaluation Criterion

Go up to Terms and Definitions

An Evaluation Criterion establishes a condition used to evaluate one or more Observations.

An Evaluation Criterion identifies the relevant observed property, state, behavior, event, or condition and establishes the value, range, tolerance, pattern, threshold, relationship, or other condition used during evaluation.

A Test Definition associates Evaluation Criteria with applicable Expected Results. DIDO-TE applies those criteria to the Observations produced during a Test Execution when determining the Test Outcome.

An Evaluation Criterion differs from an Acceptance Criterion. An Evaluation Criterion governs the evaluation of Observations. An Acceptance Criterion establishes a condition that a subject must satisfy for a designated stakeholder or authority to accept it.

An Evaluation Criterion contributes to an acceptance decision when the applicable acceptance process references that criterion.

condition used to evaluate an observation against an expected result or other specified basis

Adapted from the testing and acceptance-criteria concepts described by:

• ISO/IEC/IEEE 29119-1:2022, Software and systems engineering — Software testing — Part 1: General concepts
• ISO/IEC/IEEE 29119-2:2021, Software and systems engineering — Software testing — Part 2: Test processes
• ISTQB Glossary

An Evaluation Criterion identifies or references:

• The applicable Test Definition
• The Observation subject
• The observed property, state, behavior, event, or condition
• The applicable Expected Result or other evaluation basis
• The comparison operator or evaluation method
• The required value, range, threshold, tolerance, pattern, or relationship
• The treatment of missing, incomplete, or indeterminate Observations
• The Test Outcome associated with satisfaction or non-satisfaction of the criterion

A Test Definition may contain multiple Evaluation Criteria. The Test Definition establishes how DIDO-TE combines their individual determinations into a Test Outcome.

An Expected Result states that a Candidate Solution rejects a Transfer Bundle with an invalid integrity value.

The associated Evaluation Criteria require:

• A rejected import response
• An integrity-validation error in the applicable record
• No creation or modification of deployed resources

DIDO-TE evaluates the recorded Observations against these criteria to determine the Test Outcome.