ENV-004 — Provide Deterministic Test Environment

Go to C.2 Test Environment Requirements

The DIDO-TE SHALL provide a Test Environment in which each source of environmental variation identified by the applicable Test Definition is controlled or measured and recorded.

• [DTE5] DIDO-TE Requirements Register, source requirement identifier and obligation to be assigned
• [DTE6] Structured Information Processing Reference Architecture (SIP-RA)
• [DTE7] Federated Data Interpretation Systems Reference Architecture (FDIS-RA)

A deterministic Test Environment controls the environmental conditions capable of influencing a Test Execution or its resulting Test Results.

Relevant conditions include applied Baselines, Configuration values, Test Inputs, time sources, random values, execution ordering, resource allocation, communication conditions, and external dependencies.

Some distributed-system characteristics cannot be fixed without changing the behaviour under test. In those cases, the Test Environment must measure and record the applicable variation rather than conceal or eliminate it.

Environmental determinism does not require every Test Execution to produce an identical Test Result. It requires the DIDO-TE to control or account for environmental variation so investigators can distinguish behaviour produced by the Test Object from behaviour produced by changes in the Test Environment.

This control supports Reproducibility, Comparability, Traceability, and interpretation of Test Results.

The following child requirements decompose ENV-004 into individually verifiable obligations:

• Test Environment
• Test Definition
• Test Execution
• Test Object
• Node
• Test Input
• Test Resource
• Baseline
• Configuration
• Constraint
• Evidence

Verification confirms satisfaction of each applicable child requirement under ENV-004.

Collectively, the child requirements verify that:

• The applicable Test Definition identifies the environmental conditions requiring control.
• The DIDO-TE controls each identified condition when direct control is required and technically appropriate.
• The DIDO-TE measures and records an identified condition when controlling the condition would prevent or distort the behaviour under test.
• The DIDO-TE detects environmental conditions outside the permitted values, ranges, sequences, or tolerances.
• The DIDO-TE records controlled conditions, measured variations, detected deviations, and validation results as Evidence.
• The DIDO-TE maintains Traceability among the Test Definition, controlled or measured conditions, Test Environment, Test Execution, Evidence, and Test Results.
• The DIDO-TE prevents Test Execution when an applicable environmental condition remains uncontrolled, unmeasured, unrecorded, or outside its permitted tolerance.
• Repeated Test Executions under equivalent controlled environmental conditions provide a basis for assessing Reproducibility and Comparability.

Verification includes:

• Inspection of the applicable child-requirement verification results
• Inspection of the environmental conditions identified by the Test Definition
• Observation of environmental-condition control, measurement, and recording
• Comparison of repeated Test Executions conducted under equivalent controlled conditions
• Testing with an environmental condition outside its permitted value, range, sequence, or tolerance
• Confirmation that the DIDO-TE detects and records the variation
• Confirmation that the DIDO-TE prevents Test Execution when the applicable child requirements require prevention
• Inspection of Evidence and Traceability records

• Add links to the architecture sections governing Test Environment determinism, Configuration control, Test Input control, time, randomness, execution ordering, resource allocation, external dependencies, communication conditions, environmental monitoring, Evidence, and Test Execution admission.

Assign the applicable delivery phase.

Draft

Use the following syntax to reference this requirement’s Statement section from another DokuWiki page:

{{section>dido:03-dido-te:99-annexes:annex-c-requirements:02-test-environment-requirements:env-004-provide-deterministic-test-environment:start#Statement&noheader&nofooter&noeditbtn}}