Validation Criteria

Go up to Terms and Definitions

Validation Criteria are the conditions used to determine whether an evaluated subject satisfies a stated validation purpose.

Validation Criteria can apply to:

• A Node
• A Node Set
• A distributed system
• A Test Environment
• A Deployment Target
• An Artifact
• A Baseline
• A configuration
• A capability
• A process
• A requirement
• Another subject of Validation

Validation Criteria can establish:

• Required capabilities
• Required behaviors
• Required states
• Required state transitions
• Required outputs
• Required performance thresholds
• Required timing constraints
• Required security properties
• Required reliability properties
• Required interoperability properties
• Permitted tolerances
• Permitted exceptions
• Required Test Results
• Permitted Verdicts
• Required Evidence
• Aggregation and arbitration rules
• Conditions for a positive, negative, conditional, or inconclusive Validation Decision

Validation Criteria can identify:

• The evaluated subject
• The validation purpose
• The applicable scope
• The applicable Baseline
• The applicable Test Definitions
• The applicable Acceptance Criteria
• The required Test Results
• The permitted Verdicts
• The required Evidence
• The permitted tolerances
• The permitted exceptions
• The decision rules
• The authorized decision-maker
• The applicable Policies
• The applicable Governance Policies
• Their effective time
• Their Provenance
• Their Traceability

conditions used to determine whether an evaluated subject satisfies a stated validation purpose

Adapted from:

• Validation
• Validation Decision
• Acceptance Criteria
• ISO/IEC/IEEE 15288:2023, Systems and software engineering — System life cycle processes
• DIDO Reference Implementation Conceptual Model
• DIDO-TE draft Requirements Register

Validation Criteria differ from Acceptance Criteria:

• Validation Criteria establish the conditions used to determine whether a subject satisfies a stated validation purpose
• Acceptance Criteria establish the conditions used to determine whether an evaluated subject or outcome is acceptable

The same condition can serve as both a Validation Criterion and an Acceptance Criterion when the validation and acceptance purposes align.

Validation Criteria differ from Expected Results:

• An Expected Result specifies an anticipated outcome associated with a Sequenced Test Object or Sequence Step
• Validation Criteria determine how the relevant outcomes contribute to the Validation Decision

Validation Criteria differ from requirements:

• A requirement states an obligation that an identified subject must satisfy
• Validation Criteria establish how an authorized Actor evaluates whether the subject satisfies the stated validation purpose

Validation Criteria must be sufficiently clear, measurable, and bounded to support repeatable application.

Statements such as the following do not establish adequate Validation Criteria without additional measurable conditions:

• Performs correctly
• Operates efficiently
• Provides adequate security
• Produces acceptable results
• Supports appropriate interoperability

Validation Criteria do not need to require perfection. They can permit:

• Defined tolerances
• Known limitations
• Approved exceptions
• Conditional satisfaction
• Specified numbers or types of failures

The applicable Governance Policy must identify how those conditions affect the Validation Decision.

Validation Criteria must identify the applicable validation purpose. Criteria suitable for one intended use do not necessarily support another intended use.

Validation Criteria determine whether a selected Node Set supports a defined distributed transaction scenario.

The Validation Criteria require:

• Every required Node to join the Node Set
• Every required transaction to reach the expected committed state
• Every committed state to reference its predecessor
• No prohibited duplicate state
• Every required response to occur within five seconds
• Every required Test Result to receive a PASS Verdict
• Every Test Run to preserve the required Evidence
• No unresolved ERROR or INCONCLUSIVE Verdict

An authorized Actor applies these Validation Criteria when making the Validation Decision.