MO-004b — Validation Decision Based on DIDO Baseline Comparison

Go to MO-004 — DIDO Baseline Comparison and Validation

DIDO-TE SHALL produce a Validation Decision by evaluating the result of a selected DIDO Baseline comparison against the applicable Validation Criteria.

This requirement derives from:

• MO-004 — DIDO Baseline Comparison and Validation

The Original Requirement states:

Creating baseline results using validated nodes and comparing them against tests with modified nodes to identify discrepancies or improvements.

The source material further states:

This approach ensures that any changes or updates to the system are validated against a known benchmark, enhancing reliability and trust in the distributed system.

MO-004b preserves the portion of the Original Requirement addressing Validation based on comparison with a controlling DIDO Baseline.

The separate requirement derived from MO-004 addresses:

• MO-004a — DIDO Baseline Comparison

A DIDO Baseline comparison identifies the relationship between a selected Test Result and its corresponding Test Result in the controlling DIDO Baseline.

The comparison can identify:

• Equality
• Difference
• Direction of difference
• Magnitude of difference
• Difference within an applicable tolerance
• Difference outside an applicable tolerance
• Improvement according to defined criteria
• Degradation according to defined criteria
• Insufficient information to complete the comparison

A comparison result does not by itself establish whether the tested subject satisfies its applicable Validation Criteria.

A Validation Decision evaluates the comparison result against the applicable Validation Criteria and records the resulting determination.

The Validation Decision can determine that:

• The applicable Validation Criteria are satisfied
• The applicable Validation Criteria are not satisfied
• Available information is insufficient to make a conclusive determination

The Validation Criteria establish how the comparison result affects the Validation Decision. Different Validation Criteria can produce different Validation Decisions from the same comparison result.

For example, a measured difference of five percent can:

• Satisfy Validation Criteria permitting a ten-percent variation
• Fail to satisfy Validation Criteria permitting a two-percent variation
• Produce an inconclusive Validation Decision when the required measurement precision is unavailable

The applicable Acceptance Criteria identify:

• The selected DIDO Baseline comparison
• The subject of the Validation Decision
• The applicable Validation Criteria
• The information required to evaluate each Validation Criterion
• The permitted Validation Decision outcomes
• The rules for combining multiple Validation Criteria
• The treatment of unavailable or insufficient information
• The treatment of conflicting Validation Criteria
• The applicable Authority
• The required Evidence
• The required Provenance
• The required Traceability

MO-004b requires DIDO-TE to produce a Validation Decision. It does not authorize DIDO-TE to establish or modify the applicable Validation Criteria.

The applicable Authority, Governance Policy, or other controlling source establishes and approves the Validation Criteria.

MO-004b does not require a favorable Validation Decision. A determination that the applicable Validation Criteria are not satisfied or that available information is insufficient is a valid Validation Decision when DIDO-TE applies the controlling Validation Criteria correctly.

Separating the Validation Decision from the DIDO Baseline comparison permits independent verification of:

• The correctness of the comparison performed under MO-004a
• The correctness of the Validation Decision produced under MO-004b

This requirement applies to:

• DIDO-TE
• DIDO Baselines
• DIDO Baseline Versions
• DIDO Baseline comparisons
• DIDO Baseline comparison results
• Validation
• Validation Criteria
• Validation Decisions
• Validation subjects
• Satisfied Validation Criteria
• Unsatisfied Validation Criteria
• Inconclusive Validation Decisions
• Conflicting Validation Criteria
• Unavailable information
• Insufficient information
• Authorities
• Governance Policies
• Acceptance Criteria
• Test Definitions
• Test Environments
• Test Runs
• Test Run States
• Test Run Termination Reasons
• Incomplete Test Runs
• Test Results
• Nodes
• Node Sets
• Configurations
• Versions
• Evidence
• Provenance
• Traceability

Verification confirms that:

1. The selected DIDO Baseline comparison is identified
2. The result of the selected DIDO Baseline comparison is available
3. The Test Results participating in the DIDO Baseline comparison are identified
4. The subject of the Validation Decision is identified
5. The applicable Validation Criteria are identified
6. The source of each applicable Validation Criterion is identified
7. The applicable Authority is identified
8. The applicable Acceptance Criteria identify the information required to evaluate each Validation Criterion
9. The Acceptance Criteria identify the permitted Validation Decision outcomes
10. The Acceptance Criteria identify the rules for combining multiple Validation Criteria
11. The Acceptance Criteria identify the treatment of unavailable information
12. The Acceptance Criteria identify the treatment of insufficient information
13. The Acceptance Criteria identify the treatment of conflicting Validation Criteria
14. The Acceptance Criteria identify the treatment of comparison results derived from Incomplete Test Runs
15. DIDO-TE retrieves the result of the selected DIDO Baseline comparison
16. DIDO-TE evaluates the comparison result against each applicable Validation Criterion
17. DIDO-TE records whether the comparison result satisfies each applicable Validation Criterion
18. DIDO-TE applies the defined rules for combining multiple Validation Criteria
19. DIDO-TE produces a Validation Decision permitted by the Acceptance Criteria
20. DIDO-TE identifies the Validation Decision as conclusive or inconclusive
21. DIDO-TE records the rationale for the Validation Decision
22. DIDO-TE records each unavailable or insufficient item of information affecting the Validation Decision
23. DIDO-TE records each conflict among applicable Validation Criteria
24. DIDO-TE records the Authority and Governance Policies applicable to the Validation Decision
25. DIDO-TE preserves the Evidence supporting the Validation Decision
26. The verification record preserves Traceability among the Acceptance Criteria, applicable Authority, Governance Policies, controlling DIDO Baseline, DIDO Baseline Version, selected DIDO Baseline comparison, comparison result, participating Test Results, applicable Validation Criteria, criterion evaluations, Validation Decision, rationale, and supporting Evidence

The following pages reference this requirement:

TBD

Assess the current DIDO-TE implementation against the Validation Decision obligation established by MO-004b.

Review and accept MO-004b as a proposed derived requirement created from the evaluation and decomposition of MO-004.

The following unresolved issues affect this requirement:

Define the subject of each Validation Decision produced from a DIDO Baseline comparison.

Define how DIDO-TE identifies the applicable Validation Criteria.

Define the Authority responsible for establishing and approving the applicable Validation Criteria.

Define the permitted Validation Decision outcomes.

Define the rules for combining multiple Validation Criteria into one Validation Decision.

Define the treatment of unavailable or insufficient information.

Define the precedence and conflict-resolution rules for conflicting Validation Criteria.

Define the treatment of comparison results derived from Incomplete Test Runs for each applicable Test Run Termination Reason.

Define whether an authorized Actor must approve, affirm, or otherwise act on a Validation Decision produced by DIDO-TE.

Define the rationale DIDO-TE must record for each Validation Decision.

Define the Evidence DIDO-TE must preserve for each Validation Decision.

This requirement page should retain the stable requirement identifier MO-004b.

This page is a leaf requirement page and omits a trailing :start from its namespace.

The parent MO-004 page is a non-leaf page and retains a trailing :start in its namespace.

Changes to the Statement should preserve:

• DIDO-TE as the responsible Actor
• The result of a selected DIDO Baseline comparison as the evaluated information
• The applicable Validation Criteria as the controlling evaluation rules
• A Validation Decision as the required outcome

The Acceptance Criteria should identify:

• The selected DIDO Baseline comparison
• The result of the selected DIDO Baseline comparison
• The participating Test Results
• The subject of the Validation Decision
• The applicable Validation Criteria
• The source of each Validation Criterion
• The applicable Authority
• The applicable Governance Policies
• The permitted Validation Decision outcomes
• The information required to evaluate each Validation Criterion
• The rules for combining multiple Validation Criteria
• The treatment of unavailable information
• The treatment of insufficient information
• The treatment of conflicting Validation Criteria
• The treatment of comparison results derived from Incomplete Test Runs
• The required rationale
• The required Evidence
• The required Provenance
• The required Traceability

MO-004b does not establish or modify the applicable Validation Criteria.

MO-004b does not require a favorable Validation Decision.

A determination that the applicable Validation Criteria are not satisfied is a valid Validation Decision when DIDO-TE applies the controlling Validation Criteria correctly.

An inconclusive Validation Decision is valid when the Acceptance Criteria permit that outcome and the information required for a conclusive decision is unavailable, insufficient, invalid, or conflicting.

MO-004b does not perform the DIDO Baseline comparison. MO-004a addresses that outcome.

Material changes should receive review and should update the verification criteria, source records, parent decomposition record, and Issues section.

To reference this requirement Statement from another Wiki page, insert:

{{section>dido:03-dido-te:99-annexes:annex-c-requirements:01-mission-objectives:mo-004:mo-004b#Statement&noheader&nofooter&noeditbtn}}

Do not rename this page after an external citation unless a redirect or move plan is in place.