Evaluation Characteristic

Go up to Terms and Definitions

An Evaluation Characteristic identifies a characteristic of a subject selected for measurement, observation, comparison, or evaluation.

A subject can include:

• A Candidate Solution
• A Distributed System
• A Node
• A Node Set
• A Test Item
• A service
• A software component
• A hardware component
• A product
• A technology
• An architecture
• An implementation
• A Configuration
• An Operational Resource
• A process
• An outcome
• Another subject of evaluation

An Evaluation Characteristic can concern:

• Quality
• Functional behavior
• Performance
• Reliability
• Security
• Resource consumption
• Cost
• Compliance
• Risk
• Another observable or measurable property

An Evaluation Characteristic can identify:

• Its identity
• Its name
• Its description
• Its characteristic type
• Its subject
• Its applicable context
• Its applicable measure
• Its measurement method
• Its observation method
• Its comparison method
• Its evaluation method
• Its measurement unit
• Its required precision
• Its permitted tolerance
• Its permitted exclusions
• Its applicable reference value
• Its applicable DIDO Baseline
• Its applicable Validation Criteria
• Its applicable Acceptance Criteria
• Its applicable Governance Policies
• Its Evidence
• Its Provenance
• Its Traceability

An Evaluation Characteristic can be evaluated using:

• A direct measurement
• A derived measurement
• An observation
• A binary determination
• A qualitative classification
• A quantitative comparison
• A statistical comparison
• A structural comparison
• A behavioral comparison
• A temporal comparison
• A semantic comparison
• Another method established by the Acceptance Criteria

characteristic of a subject selected for measurement, observation, comparison, or evaluation

Adapted from:

• ISO/IEC 25010:2023, Systems and software Quality Requirements and Evaluation (SQuaRE) — Product quality model
• ISO/IEC 25002:2024, Systems and software Quality Requirements and Evaluation (SQuaRE) — Quality model overview and usage
• Candidate Solution
• Test Result
• Validation Criteria
• DIDO Reference Implementation Conceptual Model
• DIDO-TE draft Requirements Register

ISO/IEC 25002 establishes a framework for quality models composed of quality characteristics and subcharacteristics.

ISO/IEC 25010 defines a product-quality model for specifying, measuring, and evaluating ICT and software-product quality.

This definition generalizes the characteristic concept to include quality and non-quality characteristics used by DIDO-TE to evaluate Candidate Solutions and other subjects.

An Evaluation Characteristic differs from a measure:

• An Evaluation Characteristic identifies what is evaluated
• A measure identifies how a value is assigned to the Evaluation Characteristic

An Evaluation Characteristic differs from a measured value:

• An Evaluation Characteristic identifies the property subject to evaluation
• A measured value records the result obtained by applying a measure

An Evaluation Characteristic differs from Validation Criteria:

• An Evaluation Characteristic identifies what is evaluated
• Validation Criteria establish the conditions used to determine whether the evaluation outcome is acceptable

An Evaluation Characteristic differs from Acceptance Criteria:

• An Evaluation Characteristic identifies what is evaluated
• Acceptance Criteria establish the conditions, methods, limits, and evidence used to determine acceptance

The same Evaluation Characteristic can have different:

• Measures
• Measurement units
• Measurement conditions
• Reference values
• Tolerances
• Validation Criteria
• Acceptance Criteria

An Evaluation Characteristic should be sufficiently specific to permit selection of an applicable measurement, observation, comparison, or evaluation method.

Terms such as better, faster, more reliable, and more efficient do not identify objectively verifiable outcomes unless the applicable Evaluation Characteristic, method, reference, and comparison rule are established.

A Quality Characteristic is an Evaluation Characteristic concerning the degree to which a subject exhibits a quality property.

An organization uses DIDO-TE to compare three Candidate Solutions.

The applicable Acceptance Criteria select the following Evaluation Characteristics:

• Functional Correctness
• Performance Efficiency
• Reliability
• Energy Consumption
• Cost

Each Evaluation Characteristic has its own measures, measurement conditions, units, precision, tolerances, and comparison rules.

DIDO-TE preserves Traceability among each Candidate Solution, Evaluation Characteristic, measure, Test Result, comparison outcome, and supporting Evidence.