MO-004a — DIDO Baseline Comparison

Go to MO-004 — DIDO Baseline Comparison and Validation

DIDO-TE SHALL compare a selected Test Result with its corresponding Test Result in the controlling DIDO Baseline according to the Acceptance Criteria.

This requirement derives from:

• MO-004 — DIDO Baseline Comparison and Validation

The Original Requirement states:

Creating baseline results using validated nodes and comparing them against tests with modified nodes to identify discrepancies or improvements.

The source material further states:

This approach ensures that any changes or updates to the system are validated against a known benchmark, enhancing reliability and trust in the distributed system.

MO-004a preserves the portion of the Original Requirement addressing comparison of selected Test Results with Test Results in a controlling DIDO Baseline.

The separate requirement derived from MO-004 addresses:

• MO-004b — Validation Decision Based on DIDO Baseline Comparison

A DIDO Baseline provides an approved reference for evaluating the behavior, characteristics, and outcomes of a selected Distributed System.

Comparison with the controlling DIDO Baseline permits an organization to identify:

• Equality
• Difference
• Direction of difference
• Magnitude of difference
• Difference within an applicable tolerance
• Difference outside an applicable tolerance
• Improvement according to defined criteria
• Degradation according to defined criteria
• Insufficient information to complete the comparison

A comparison requires DIDO-TE to identify both:

• The selected Test Result
• The corresponding Test Result in the controlling DIDO Baseline

The Test Results correspond when the Acceptance Criteria establish that they represent the same characteristic, measurement, observation, event, output, or other comparison subject under the applicable comparison conditions.

The Acceptance Criteria establish:

• The controlling DIDO Baseline
• The controlling DIDO Baseline Version
• The Test Result selected for comparison
• The corresponding Test Result in the controlling DIDO Baseline
• The characteristic subject to comparison
• The comparison method
• The measurement conditions
• The measurement units
• The required measurement precision
• The permitted tolerances
• The permitted exclusions
• The treatment of missing values
• The treatment of invalid values
• The treatment of nondeterministic behavior
• The classifications assigned to comparison outcomes
• The treatment of Test Results produced by Incomplete Test Runs

The comparison can be:

• Quantitative
• Qualitative
• Structural
• Behavioral
• Temporal
• Semantic
• Binary
• Statistical
• Another comparison method established by the Acceptance Criteria

MO-004a requires comparison but does not require DIDO-TE to determine whether the comparison outcome satisfies applicable Validation Criteria.

MO-004b addresses the resulting Validation Decision.

Separating comparison from the Validation Decision permits:

• One comparison outcome to support multiple Validation Decisions
• Different Validation Criteria to evaluate the same comparison outcome
• An authorized Actor to review the comparison before making a Validation Decision
• DIDO-TE to preserve a comparison outcome even when the available information is insufficient for a conclusive Validation Decision

This requirement applies to:

• DIDO-TE
• DIDO Baselines
• DIDO Baseline Versions
• Selected Test Results
• Corresponding Test Results
• Test Result characteristics
• Comparison methods
• Comparison conditions
• Measurement units
• Measurement precision
• Permitted tolerances
• Permitted exclusions
• Equality
• Difference
• Direction of difference
• Magnitude of difference
• Improvement
• Degradation
• Inconclusive comparisons
• Missing values
• Invalid values
• Nondeterministic behavior
• Test Definitions
• Test Environments
• Test Runs
• Test Run States
• Test Run Termination Reasons
• Incomplete Test Runs
• Nodes
• Node Sets
• Configurations
• Versions
• Acceptance Criteria
• Evidence
• Provenance
• Traceability

Verification confirms that:

1. The controlling DIDO Baseline is identified
2. The controlling DIDO Baseline Version is identified
3. The selected Test Result is identified
4. The corresponding Test Result in the controlling DIDO Baseline is identified
5. The Acceptance Criteria establish the correspondence between the selected Test Result and the Test Result in the controlling DIDO Baseline
6. The Acceptance Criteria identify each Test Result characteristic subject to comparison
7. The Acceptance Criteria establish the comparison method for each compared characteristic
8. The Acceptance Criteria establish the applicable comparison conditions
9. The Acceptance Criteria establish the applicable measurement units
10. The Acceptance Criteria establish the required measurement precision
11. The Acceptance Criteria establish the permitted tolerances
12. The Acceptance Criteria establish the permitted exclusions
13. The Acceptance Criteria establish the treatment of missing values
14. The Acceptance Criteria establish the treatment of invalid values
15. The Acceptance Criteria establish the treatment of nondeterministic behavior
16. The Acceptance Criteria establish the classifications applicable to comparison outcomes
17. The Acceptance Criteria establish the treatment of Test Results produced by Incomplete Test Runs
18. DIDO-TE retrieves the selected Test Result
19. DIDO-TE retrieves the corresponding Test Result from the controlling DIDO Baseline
20. DIDO-TE applies the comparison method established by the Acceptance Criteria
21. DIDO-TE compares each selected Test Result characteristic with its corresponding DIDO Baseline Test Result characteristic
22. DIDO-TE determines the difference between each compared characteristic
23. DIDO-TE determines whether each difference is within the permitted tolerance
24. DIDO-TE classifies each comparison outcome according to the Acceptance Criteria
25. DIDO-TE records each comparison outcome
26. DIDO-TE records any Test Result characteristic it cannot compare
27. DIDO-TE records the reason it cannot complete a comparison
28. The verification record preserves Traceability among the Acceptance Criteria, controlling DIDO Baseline, DIDO Baseline Version, selected Test Result, corresponding DIDO Baseline Test Result, compared characteristics, comparison method, measured differences, permitted tolerances, comparison outcomes, and supporting Evidence

The following pages reference this requirement:

TBD

Assess the current DIDO-TE implementation against the DIDO Baseline comparison obligation established by MO-004a.

Review and accept MO-004a as a proposed derived requirement created from the evaluation and decomposition of MO-004.

The following unresolved issues affect this requirement:

Determine whether DIDO Baseline requires a separate shared Terms and Definitions entry or identifies a DIDO-specific application of Baseline.

Determine whether Baseline Test Result requires a separate shared Terms and Definitions entry.

Define how DIDO-TE identifies the controlling DIDO Baseline and DIDO Baseline Version.

Define how DIDO-TE establishes correspondence between a selected Test Result and a Test Result in the controlling DIDO Baseline.

Define the Test Result characteristics subject to comparison.

Define the supported quantitative, qualitative, structural, behavioral, temporal, semantic, binary, and statistical comparison methods.

Define the measurement conditions, units, precision, exclusions, and tolerances used for each comparison method.

Define the classifications assigned to equality, difference, improvement, degradation, and inconclusive comparison outcomes.

Define the treatment of missing, invalid, incompatible, or unavailable Test Results.

Define the treatment of nondeterministic Test Results.

Define the treatment of Test Results produced by Incomplete Test Runs for each applicable Test Run Termination Reason.

Define the Evidence DIDO-TE must preserve for each comparison outcome.

This requirement page should retain the stable requirement identifier MO-004a.

This page is a leaf requirement page and omits a trailing :start from its namespace.

The parent MO-004 page is a non-leaf page and retains a trailing :start in its namespace.

Changes to the Statement should preserve:

• DIDO-TE as the responsible Actor
• A selected Test Result as the subject of comparison
• The corresponding Test Result in the controlling DIDO Baseline as the comparison reference
• The Acceptance Criteria as the source of the comparison rules

The Acceptance Criteria should identify:

• The controlling DIDO Baseline
• The controlling DIDO Baseline Version
• The selected Test Result
• The corresponding Test Result in the controlling DIDO Baseline
• The characteristics subject to comparison
• The comparison method
• The comparison conditions
• The measurement units
• The required measurement precision
• The permitted tolerances
• The permitted exclusions
• The treatment of missing values
• The treatment of invalid values
• The treatment of incompatible values
• The treatment of unavailable Test Results
• The treatment of nondeterministic behavior
• The classifications applicable to comparison outcomes
• The treatment of Test Results produced by Incomplete Test Runs
• The required Evidence
• The required Provenance
• The required Traceability

MO-004a does not determine whether a comparison outcome satisfies applicable Validation Criteria.

MO-004a does not produce the resulting Validation Decision. MO-004b addresses that outcome.

Material changes should receive review and should update the verification criteria, source records, parent decomposition record, and Issues section.

To reference this requirement Statement from another Wiki page, insert:

{{section>dido:03-dido-te:99-annexes:annex-c-requirements:01-mission-objectives:mo-004:mo-004a#Statement&noheader&nofooter&noeditbtn}}

Do not rename this page after an external citation unless a redirect or move plan is in place.