ENV-007 — Support Repeatable Test Environments

Go to C.2 Test Environment Requirements

The DIDO-TE SHALL reproduce a specified Test Environment across applicable Test Executions.

• [DTE5] DIDO-TE Requirements Register, source requirement identifier and obligation to be assigned
• [DTE6] Structured Information Processing Reference Architecture (SIP-RA)
• [DTE7] Federated Data Interpretation Systems Reference Architecture (FDIS-RA)

Repeatable Test Environments permit multiple Test Executions to begin from equivalent defined conditions. Repeatability supports comparison among Test Results, reproduction of observed behaviour, investigation of failures, confirmation of corrective actions, regression testing, and independent reassessment.

Repeatability does not require every execution to produce identical Test Results. Concurrent processing, distributed communication, external dependencies, timing variation, randomized inputs, resource contention, and Test Object behavior may produce legitimate differences. The applicable Test Definition identifies the environmental characteristics that must remain equivalent, the characteristics permitted to vary, and the tolerances governing acceptable variation.

A repeatable Test Environment depends on controlled descriptions of its resources, Nodes, software, configurations, dependencies, data, credentials, network conditions, time sources, external services, security controls, initial states, and provisioning procedures. The DIDO-TE preserves the identity and version of each controlled element and records substitutions, exceptions, and deviations.

The DIDO-TE compares the Test Environment established for each Test Execution with the specified environmental baseline. This comparison establishes whether differences among Test Results arise from the Test Object, specified test variation, or unintended variation in the Test Environment.

• Test Environment
• Test Definition
• Test Execution
• Test Object
• Node
• Test Resource
• Configuration
• Constraint
• Dependency
• Evidence
• Test Result

Verification confirms that:

• The applicable Test Definition identifies the Test Environment subject to reproduction.
• The Test Definition identifies each environmental characteristic requiring repeatability.
• Each environmental characteristic requiring repeatability has a unique identity.
• The Test Definition identifies the environmental characteristics permitted to vary among Test Executions.
• The Test Definition specifies the permitted range, tolerance, distribution, sequence, timing, or other variation for each variable characteristic.
• The Test Definition identifies each Test Resource, Node, software component, executable, configuration, dependency, data set, credential, service, interface, communication path, and security control material to environmental repeatability.
• The Test Definition specifies the required identity, version, revision, digest, configuration, state, or other distinguishing information for each controlled environmental element.
• The Test Definition specifies the required initial state of the Test Environment and Test Object.
• The Test Definition specifies