ENV-007 — Support Repeatable Test Environments [Dido Solutions, Inc Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== ENV-007 — Support Repeatable Test Environments ======

[[dido:03-dido-te:99-annexes:annex-c-requirements:02-test-environment-requirements:start|Go to C.2 Test Environment Requirements]]

===== Statement =====

The [[dido:99_annexes:annex-b-terms-and-definitions:d:dido-te|DIDO-TE]] SHALL reproduce a specified [[dido:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]] across applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution|Test Executions]].

===== Derived From =====

  * [[dido:03-dido-te:99-annexes:annex-b-references:dte-005|[DTE5] DIDO-TE Requirements Register]], source requirement identifier and obligation to be assigned
  * [[dido:03-dido-te:99-annexes:annex-b-references:dte-006|[DTE6] Structured Information Processing Reference Architecture (SIP-RA)]]
  * [[dido:03-dido-te:99-annexes:annex-b-references:dte-007|[DTE7] Federated Data Interpretation Systems Reference Architecture (FDIS-RA)]]

===== Rationale =====

Repeatable Test Environments permit multiple Test Executions to begin from equivalent defined conditions. Repeatability supports comparison among Test Results, reproduction of observed behaviour, investigation of failures, confirmation of corrective actions, regression testing, and independent reassessment.

Repeatability does not require every execution to produce identical Test Results. Concurrent processing, distributed communication, external dependencies, timing variation, randomized inputs, resource contention, and Test Object behavior may produce legitimate differences. The applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_definition|Test Definition]] identifies the environmental characteristics that must remain equivalent, the characteristics permitted to vary, and the tolerances governing acceptable variation.

A repeatable Test Environment depends on controlled descriptions of its resources, Nodes, software, configurations, dependencies, data, credentials, network conditions, time sources, external services, security controls, initial states, and provisioning procedures. The DIDO-TE preserves the identity and version of each controlled element and records substitutions, exceptions, and deviations.

The DIDO-TE compares the Test Environment established for each Test Execution with the specified environmental baseline. This comparison establishes whether differences among Test Results arise from the Test Object, specified test variation, or unintended variation in the Test Environment.

===== Applies To =====

  * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environment]]
  * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_definition|Test Definition]]
  * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution|Test Execution]]
  * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_object|Test Object]]
  * [[dido:99_annexes:annex-b-terms-and-definitions:n:node|Node]]
  * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_resource|Test Resource]]
  * [[dido:99_annexes:annex-b-terms-and-definitions:c:configuration|Configuration]]
  * [[dido:99_annexes:annex-b-terms-and-definitions:c:constraint|Constraint]]
  * [[dido:99_annexes:annex-b-terms-and-definitions:d:dependency|Dependency]]
  * [[dido:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]]
  * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]]

===== Verification =====

Verification confirms that:

  * The applicable Test Definition identifies the Test Environment subject to reproduction.
  * The Test Definition identifies each environmental characteristic requiring repeatability.
  * Each environmental characteristic requiring repeatability has a unique identity.
  * The Test Definition identifies the environmental characteristics permitted to vary among Test Executions.
  * The Test Definition specifies the permitted range, tolerance, distribution, sequence, timing, or other variation for each variable characteristic.
  * The Test Definition identifies each Test Resource, Node, software component, executable, configuration, dependency, data set, credential, service, interface, communication path, and security control material to environmental repeatability.
  * The Test Definition specifies the required identity, version, revision, digest, configuration, state, or other distinguishing information for each controlled environmental element.
  * The Test Definition specifies the required initial state of the Test Environment and Test Object.
  * The Test Definition specifies