Differences

This shows you the differences between two versions of the page.

--- dido:03-dido-te:99-annexes:annex-c-requirements:01-mission-objectives:mo-002:start [2026/08/05 11:09] – removed - external edit (Unknown date) 127.0.0.1
+++ dido:03-dido-te:99-annexes:annex-c-requirements:01-mission-objectives:mo-002:start [2026/08/06 07:29] (current) – nick_dido
@@ Line 1: / Line 1: @@
+====== MO-002 — Reproducible Test Environments and Test Execution ======
+[[dido:03-dido-te:99-annexes:annex-c-requirements:01-mission-objectives:start|Go to C.1 Mission Objectives]]
+===== Original Requirement =====
+The draft DIDO-TE source material states:
+> //Create repeatable testing scenarios, including regression testing, that allow rigorous evaluation of distributed systems under different conditions and configurations.//
+<todo>Identify the authoritative source, original requirement identifier, and controlling citation for this Original Requirement.</todo>
+===== Assessment of Original Requirement =====
+The Original Requirement preserves the approved mission-level intent that [[dido:99_annexes:annex-b-terms-and-definitions:d:dido-te|DIDO-TE]] provide repeatable or reproducible testing under different conditions and [[dido:99_annexes:annex-b-terms-and-definitions:c:configuration|Configurations]].
+The Original Requirement does not express an independently interpretable or independently verifiable normative obligation.
+The following Specification Discipline and Authoring findings apply:
+  * **Create** does not identify an explicit responsible [[dido:99_annexes:annex-b-terms-and-definitions:a:actor|Actor]]
+  * **Create** does not use an explicit normative modal
+  * **Repeatable** does not identify the required degree or method of [[dido:99_annexes:annex-b-terms-and-definitions:r:reproducibility|Reproducibility]]
+  * **Testing scenarios** does not use defined terms such as [[dido:99_annexes:annex-b-terms-and-definitions:t:test_definition|Test Definition]], [[dido:99_annexes:annex-b-terms-and-definitions:t:test_sequence|Test Sequence]], [[dido:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]], or [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution|Test Execution]]
+  * **Including regression testing** provides an example but does not establish a bounded scope
+  * **Allow** is a weak verb that does not identify an observable DIDO-TE behavior
+  * **Rigorous evaluation** is subjective and does not identify measurable evaluation criteria
+  * **Distributed systems** does not identify the selected [[dido:99_annexes:annex-b-terms-and-definitions:d:distributed_system|Distributed System]], [[dido:99_annexes:annex-b-terms-and-definitions:t:test_item|Test Items]], [[dido:99_annexes:annex-b-terms-and-definitions:n:node|Nodes]], or [[dido:99_annexes:annex-b-terms-and-definitions:n:node_set|Node Sets]] subject to testing
+  * **Different conditions** does not identify the controlled conditions, variable conditions, or permitted differences
+  * **Different configurations** does not identify the Configurations that DIDO-TE must preserve, recreate, or vary
+  * The Original Requirement does not identify the applicable [[dido:99_annexes:annex-b-terms-and-definitions:a:acceptance_criteria|Acceptance Criteria]]
+  * The Original Requirement does not identify the [[dido:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]] required to demonstrate Reproducibility
+The source uses the word **repeatable**, while the proposed canonical requirements use **reproducible** and **Reproducibility**. The requirement owner must confirm whether:
+  * Repeatability and Reproducibility express the same approved intent in this context
+  * Repeatability identifies a narrower controlled-condition obligation
+  * Reproducibility identifies a broader obligation that permits defined changes to operators, resources, locations, or other conditions
+  * Separate requirements for Repeatability and Reproducibility are necessary
+The Original Requirement combines at least two independently verifiable outcomes:
+  * Recreating a selected Test Environment within an established permitted variation
+  * Repeating selected Test Execution within an established permitted variation
+These outcomes require separate verification.
+A Test Environment can be recreated successfully even when repeated Test Execution produces unacceptable differences in the resulting [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]].
+Test Execution can also produce acceptable Test Results even when DIDO-TE has not demonstrated that the Test Environment was recreated according to its controlling Configuration.
+The Original Requirement therefore requires:
+  * Identification of DIDO-TE as the responsible Actor
+  * Replacement of nonnormative language with explicit normative obligations
+  * Selection of Repeatability, Reproducibility, or both as the controlling quality
+  * Definition of the Test Environment characteristics subject to reproduction
+  * Definition of the Test Execution information subject to reproduction
+  * Separation of Test Environment Reproducibility from Test Execution Reproducibility
+  * Identification of the permitted variation for each outcome
+  * Identification of the applicable Acceptance Criteria
+  * Identification of the Evidence required to verify each outcome
+  * Confirmation that the derived requirements collectively preserve the complete intent of MO-002
+The decomposition preserves ''MO-002'' as the stable parent requirement identifier. Each proposed replacement requirement receives a lettered identifier and a separate leaf requirement page.
+===== Proposed Statements =====
+The Original Requirement is decomposed into the following proposed replacement requirements:
+{{indexmenu>dido:03-dido-te:99-annexes:annex-c-requirements:01-mission-objectives:mo-002#1|js navbar nocookie maxjs#1 id#dido_te_mo_002_requirements_nav}}
+===== Requirement Status =====
+<todo>Review and approve the decomposition of MO-002 into MO-002a and MO-002b.</todo>
+<todo>Confirm whether Repeatability, Reproducibility, or separate requirements for both qualities preserve the approved intent of the Original Requirement.</todo>
+<todo>Determine whether MO-002a and MO-002b collectively supersede MO-002.</todo>
+<todo>If MO-002a and MO-002b are accepted as the complete replacement for MO-002, mark MO-002 as superseded and preserve this page as the parent [[dido:99_annexes:annex-b-terms-and-definitions:t:traceability|Traceability]] record.</todo>
+----
+===== Issues =====
+The following unresolved issues affect the decomposition of MO-002:
+<todo>Identify the authoritative source, original requirement identifier, and controlling citation for the Original Requirement.</todo>
+<todo>Confirm the relationship between Repeatability and [[dido:99_annexes:annex-b-terms-and-definitions:r:reproducibility|Reproducibility]] within DIDO-TE.</todo>
+<todo>Determine whether Repeatability requires a separate shared Terms and Definitions entry.</todo>
+<todo>Define the controlled and variable characteristics of a reproducible [[dido:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]].</todo>
+<todo>Define the [[dido:99_annexes:annex-b-terms-and-definitions:c:configuration|Configuration]], [[dido:99_annexes:annex-b-terms-and-definitions:v:version|Versions]], [[dido:99_annexes:annex-b-terms-and-definitions:d:dependency|Dependencies]], [[dido:99_annexes:annex-b-terms-and-definitions:o:operational_resource|Operational Resources]], and [[dido:99_annexes:annex-b-terms-and-definitions:d:deployment_target|Deployment Targets]] that DIDO-TE must preserve or recreate.</todo>
+<todo>Define the [[dido:99_annexes:annex-b-terms-and-definitions:t:test_definition|Test Definitions]], [[dido:99_annexes:annex-b-terms-and-definitions:t:test_sequence|Test Sequences]], [[dido:99_annexes:annex-b-terms-and-definitions:s:sequence_step|Sequence Steps]], [[dido:99_annexes:annex-b-terms-and-definitions:t:test_argument_value|Test Argument Values]], [[dido:99_annexes:annex-b-terms-and-definitions:s:starting_condition|Starting Conditions]], [[dido:99_annexes:annex-b-terms-and-definitions:t:timing_condition|Timing Conditions]], and [[dido:99_annexes:annex-b-terms-and-definitions:e:execution_path|Execution Paths]] that DIDO-TE must preserve for reproducible Test Execution.</todo>
+<todo>Define whether Reproducibility requires exact equality or permits bounded variation.</todo>
+<todo>Define the permitted variation for Test Environment recreation.</todo>
+<todo>Define the permitted variation for repeated Test Execution and resulting [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]].</todo>
+<todo>Define the comparison methods, tolerances, precision, and treatment of nondeterministic behavior.</todo>
+<todo>Define how failed, incomplete, interrupted, timed-out, canceled, and repeated [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Runs]] affect the determination of Reproducibility.</todo>
+<todo>Define the [[dido:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]], [[dido:99_annexes:annex-b-terms-and-definitions:p:provenance|Provenance]], and Traceability required to demonstrate Test Environment and Test Execution Reproducibility.</todo>
+<todo>Determine whether MO-002a and MO-002b provide complete coverage of the approved intent of MO-002.</todo>
+----
+===== Notes for Editors =====
+This page should retain the stable parent requirement identifier ''MO-002''.
+This page is a non-leaf requirement page and retains a trailing '':start'' in its namespace.
+The child requirements are leaf requirement pages and omit a trailing '':start'' from their namespaces.
+This page preserves:
+  * The Original Requirement
+  * The assessment of the Original Requirement
+  * The reason for decomposition
+  * Traceability to the proposed replacement requirements
+  * The unresolved cross-cutting issues affecting the decomposition
+  * The supersession decision
+MO-002a addresses the Reproducibility of a selected [[dido:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]].
+MO-002b addresses the Reproducibility of selected [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution|Test Execution]].
+The child requirements must not combine Test Environment recreation and Test Execution repetition into a single normative Statement.
+The child requirements should identify independently verifiable [[dido:99_annexes:annex-b-terms-and-definitions:a:acceptance_criteria|Acceptance Criteria]] for:
+  * The selected Test Environment
+  * The selected Test Execution
+  * The controlling Configurations
+  * The controlling Versions
+  * The applicable Test Argument Values
+  * The Starting Conditions
+  * The Timing Conditions
+  * The permitted Execution Paths
+  * The permitted variation
+  * The comparison method
+  * The treatment of nondeterministic behavior
+  * The required Evidence
+  * The required Provenance
+  * The required Traceability
+MO-002 should not be marked as superseded until the requirement owner:
+  * Confirms the authoritative Original Requirement and its controlling citation
+  * Approves the interpretation of Repeatability and Reproducibility
+  * Approves MO-002a
+  * Approves MO-002b
+  * Confirms that the child requirements collectively preserve the complete approved intent of MO-002
+  * Confirms that no additional child requirements are required
+After supersession, this page should remain as the parent Traceability record and should continue to preserve the Original Requirement and its assessment.
+Material changes to the decomposition should update the child requirements, Requirement Status, Issues, source records, and Traceability records.
+Do not rename this page after an external citation unless a redirect or move plan is in place.
+----
+<WRAP centeralign>
+© 2026 Dido Solutions, Inc. and Jackrabbit Consulting, Inc.
+</WRAP>