Reliability

Go up to Terms and Definitions

Reliability is a Quality Characteristic concerning the degree to which a subject performs specified functions under specified conditions for a specified period.

Reliability can apply to:

• A Candidate Solution
• A Distributed System
• A Node
• A Qualified Node
• A Node Set
• A service
• A software component
• A hardware component
• An Execution Facility
• An Operational Resource
• Another subject required to perform specified functions

Reliability does not mean that a subject never fails. Reliability concerns whether the subject satisfies defined expectations for operation under stated conditions for a stated period.

Reliability can concern:

• Availability
• Continuity of operation
• Failure frequency
• Failure rate
• Fault tolerance
• Degraded operation
• Recovery
• State preservation
• Information preservation
• Evidence continuity
• Successful completion
• Service-level behavior
• Another reliability-related property

Reliability can identify:

• The evaluated subject
• The specified functions
• The specified operating conditions
• The evaluation period
• The applicable workload
• The applicable Test Environment
• The applicable Configuration
• The applicable failure conditions
• The applicable fault conditions
• The applicable failure classifications
• The applicable Reliability measures
• The applicable measurement units
• The required measurement precision
• The permitted tolerance
• The applicable DIDO Baseline
• The applicable Validation Criteria
• The applicable Acceptance Criteria
• Its Evidence
• Its Provenance
• Its Traceability

Reliability can be evaluated using measures such as:

• Successful operations per attempted operation
• Successful Test Runs per attempted Test Run
• Failures per unit of time
• Failures per number of operations
• Mean time between failures
• Mean time to failure
• Availability during the evaluation period
• Successful recovery attempts per recovery attempt
• Information or state preserved after a failure
• Evidence continuity after a failure or restart
• Another measure established by the Acceptance Criteria

Within the Governed Node Service Market, Reliability supports:

• Trust in repeated governed work
• Service continuity
• Evidence continuity
• Provider comparison
• Competitive Substitution

A Qualified Node demonstrates Reliability by performing its governed functions according to the applicable conditions, service expectations, Governance Policies, and Acceptance Criteria.

Quality Characteristic concerning the degree to which a subject performs specified functions under specified conditions for a specified period

Adapted from:

• ISO/IEC 25010:2023, Systems and software Quality Requirements and Evaluation (SQuaRE) — Product quality model
• ISO/IEC 25023:2016, Systems and software Quality Requirements and Evaluation (SQuaRE) — Measurement of system and software product quality
• Financial Systems Archetype, Part 7: Governed Node Service Market
• Quality Characteristic
• DIDO Reference Implementation Conceptual Model
• DIDO-TE draft Requirements Register

The original shared definition described Reliability as the ability of a Qualified Node to perform a governed function correctly and consistently under stated operating conditions.

This revision generalizes Reliability for use throughout the shared DIDO corpus while preserving the original Qualified Node and Governed Node Service Market application.

Reliability differs from Availability:

• Availability concerns whether a subject is operational and accessible when required
• Reliability concerns whether a subject performs specified functions under specified conditions for a specified period

Availability can contribute to a Reliability evaluation, but Availability and Reliability are not interchangeable.

Reliability differs from Functional Correctness:

• Reliability concerns sustained performance of specified functions
• Functional Correctness concerns whether produced results are correct with the required precision

A subject can perform an incorrect function consistently.

A subject can produce correct results during successful operations while failing too frequently to satisfy its Reliability criteria.

Reliability differs from Operational Stability:

• Reliability concerns performance of specified functions over a specified period
• Operational Stability concerns whether selected operational characteristics remain within permitted variation

Reliability differs from Recoverability:

• Reliability concerns performance under specified conditions for a specified period
• Recoverability concerns restoration of an affected state and required information after an interruption or failure

Recoverability can contribute to Reliability but does not replace Reliability.

A completed Test Run assigned a failing Verdict does not necessarily indicate a Reliability failure. The Verdict can concern Functional Correctness, Performance Efficiency, Security, Interoperability, or another Evaluation Characteristic.

A Test Execution Exception does not necessarily indicate failure of the evaluated subject. The exception can originate in:

• DIDO-TE
• The Test Environment
• An Execution Facility
• An Operational Resource
• An external system
• Another supporting subject

The applicable Acceptance Criteria should distinguish:

• Failure of the evaluated subject
• Failure of DIDO-TE
• Failure of the Test Environment
• Failure of a supporting Operational Resource
• A Test Execution Exception unrelated to the evaluated subject

A persistence Node records Evidence references and retention metadata during normal operating conditions and after a controlled restart.

The applicable Acceptance Criteria establish:

• The specified persistence functions
• The operating conditions
• The evaluation period
• The permitted failure frequency
• The required availability
• The required recovery behavior
• The information and Evidence that must remain intact
• The permitted recovery duration

The Node exhibits Reliability when it performs the specified functions according to those conditions and limits throughout the evaluation period.

In a separate DIDO-TE evaluation, three Candidate Solutions perform the same function for 72 hours under a common workload. DIDO-TE records each attempted operation, successful operation, failure, period of unavailability, recovery attempt, Test Execution Exception, and supporting Evidence.

The resulting Test Results support a comparative evaluation of Candidate Solution Reliability.