Test Environment

Go up to Terms and Definitions

A Test Environment identifies the configured Test Items, resources, conditions, and controls within which a test is executed.

A Test Environment establishes the execution context required by a Test Definition. The Test Environment can identify:

• The System Under Test
• Supporting Test Items
• Nodes
• Node Sets
• Deployment Targets
• Computing resources
• Storage resources
• Network topology
• Communication paths
• Software
• Configuration settings
• Test data
• Credentials and authorization references
• Security controls
• Operational conditions
• State conditions
• Time conditions
• Observation points
• Measurement facilities
• Evidence-collection facilities
• Applicable Governance Policies

A Test Environment can reference reusable environment definitions, Node Profiles, Deployment Target records, Test Items, and other controlled artifacts.

A Test Environment can exist within a:

• Connected Environment
• Disconnected Environment
• Air-Gapped Environment

The Test Environment definition identifies the required configuration. A realized Test Environment identifies the resources and conditions actually used for a Test Run.

The distinction permits an evaluator to compare the required Test Environment with the realized Test Environment and determine whether the Test Run occurred under the required conditions.

configured combination of Test Items, resources, conditions, and controls within which a test is executed

Adapted from:

• OMG Test Information Interchange Format (TestIF), Version 1.0 Beta 3, Section 7.3.6, TestEnvironment
• DIDO Test Environment System Description, Version 0.2
• Distributed Immutable Data Object Test Environment (DIDO-TE)
• DIDO-TE draft Requirements Register
• United States Patent Application Publication US 2022/0237111 A1, Distributed Immutable Data Object Testing
• United States Patent US 12,086,057 B2, Distributed Immutable Data Object Testing

OMG TestIF describes a Test Environment as a Test Object that identifies the system configuration necessary to execute a test. The DIDO-TE definition extends the configuration concept to identify the Test Items, resources, conditions, and controls required for reproducible testing and validation.

A Test Environment is not the same as DIDO-TE.

DIDO-TE identifies the governed testing system. A Test Environment identifies a configured execution context managed or used by that system.

One DIDO-TE implementation can manage multiple Test Environments.

A Test Environment does not necessarily contain only virtual resources. A Test Environment can include:

• Physical Nodes
• Virtual Nodes
• Containers
• Simulated Nodes
• Emulated Nodes
• External systems
• Cloud resources
• On-premises resources
• Test instruments
• Human-operated Test Items

A Test Environment differs from a Deployment Target:

• A Deployment Target identifies a specific configured destination to which a Deployment can deliver an Infrastructure Environment
• A Test Environment identifies the complete configured context within which a test is executed

A Test Environment can use one or more Deployment Targets.

A Test Environment definition does not demonstrate that the required configuration existed during a Test Run. Test Evidence must identify the realized configuration and applicable execution conditions.

A Test Environment for evaluating a DIDO node implementation identifies:

• Four validated reference Nodes
• One changed Node
• The applicable Node Profiles
• The selected Deployment Targets
• The virtual network topology
• The permitted communication paths
• The initial DIDO state
• The required clock conditions
• The Test Data
• The observation points
• The Evidence-collection facilities
• The applicable Governance Policies
• The security and authorization controls

DIDO-TE realizes the Test Environment before initiating the Test Run. The Test Evidence records the resources, configurations, versions, and conditions actually present during execution.