MO-003 — Automated Test Execution

Go to C.1 Mission Objectives

DIDO-TE SHALL complete a selected Test Execution without human intervention after initiation.

This requirement consolidates mission-level intent from the DIDO-TE source material addressing:

• Automated execution of testing scenarios
• Automated performance of Test Sequences
• Coordinated performance of testing activities across Nodes
• Delivery of recorded inputs to Nodes
• Collection of outputs from Nodes
• Automated generation of Test Results
• Preservation of supporting Evidence

Identify the authoritative source requirements, original requirement identifiers, and controlling citations for the automated Test Execution objective.

MO-003 expresses the common mission-level outcome. Detailed obligations addressing Test Execution initiation, sequencing, scheduling, coordination, monitoring, exception handling, termination, Test Result generation, and Evidence preservation remain separate Operational, Functional, Logging and Auditing, Reliability, Performance, and Data Management Requirements.

Automated Test Execution permits organizations to perform complex and repeated testing consistently without requiring a human Actor to perform each Test Step.

Automation supports:

• Consistent performance of Test Definitions
• Coordinated performance of Test Sequences
• Performance of Test Steps in the defined order
• Application of Test Argument Values
• Evaluation of applicable Starting Conditions
• Evaluation of applicable Timing Conditions
• Selection of permitted Execution Paths
• Enforcement of applicable Timeouts
• Coordination among Nodes and Node Sets
• Collection of observations and measurements
• Generation of Test Results
• Preservation of Evidence
• Repetition of testing for regression analysis
• Performance of testing in connected, disconnected, and air-gapped environments

The phrase without human intervention after initiation establishes the automation boundary for MO-003.

MO-003 does not prohibit a human Actor from:

• Creating or selecting a Test Definition
• Selecting a Test Environment
• Supplying or approving Test Argument Values
• Establishing Acceptance Criteria
• Authorizing Test Execution
• Initiating a Test Run
• Monitoring Test Execution
• Requesting cancellation
• Reviewing Test Results
• Making a Validation Decision

After initiation, DIDO-TE performs the selected Test Execution without requiring a human Actor to perform a Test Step, select an Execution Path, transfer an intermediate result, or advance the Test Sequence.

An authorized human request to pause, cancel, or otherwise control a Test Run does not by itself demonstrate a failure of automated Test Execution when the applicable Acceptance Criteria permit the requested operation.

The detailed requirements establish:

• The event constituting initiation
• The event constituting completion
• The permitted pre-execution human activities
• The permitted human control operations after initiation
• The applicable Test Definition
• The applicable Test Environment
• The automated Test Steps
• The required Test Executables
• The required Execution Facilities
• The applicable exception-handling behavior
• The required Test Results and Evidence

This requirement applies to:

• DIDO-TE
• Selected Test Execution
• Automated Test Execution
• Acceptance Criteria
• Actors
• Human Actors
• Users
• Test Definitions
• Test Sequences
• Sequence Steps
• Test Steps
• Test Executables
• Executable Instructions
• Executable Artifacts
• Execution Facilities
• Test Arguments
• Test Argument Values
• Starting Conditions
• Timing Conditions
• Execution Paths
• Timeouts
• Test Environments
• Nodes
• Node Sets
• Test Runs
• Test Results
• Test Run States
• Test Run Termination Reasons
• Incomplete Test Runs
• Connected Environments
• Disconnected Environments
• Air-Gapped Environments
• Evidence
• Provenance
• Traceability

Verification confirms that:

1. The selected Test Execution is identified
2. The applicable Test Definition and Version are identified
3. The selected Test Environment is identified
4. The Acceptance Criteria identify the event constituting Test Execution initiation
5. The Acceptance Criteria identify the event constituting Test Execution completion
6. The Acceptance Criteria identify the human activities permitted before initiation
7. The Acceptance Criteria identify the human control operations permitted after initiation
8. The Acceptance Criteria identify the Test Steps subject to automated performance
9. The Acceptance Criteria identify the applicable Test Executables
10. The Acceptance Criteria identify the applicable Execution Facilities
11. The Acceptance Criteria identify the applicable Test Argument Values
12. The Acceptance Criteria identify the applicable Starting Conditions
13. The Acceptance Criteria identify the applicable Timing Conditions
14. The Acceptance Criteria identify the permitted Execution Paths
15. The Acceptance Criteria identify the applicable Timeouts
16. An authorized Actor initiates the selected Test Execution
17. DIDO-TE performs each automated Test Step after initiation
18. DIDO-TE advances the applicable Test Sequence without requiring human intervention
19. DIDO-TE transfers required intermediate information without requiring human intervention
20. DIDO-TE evaluates applicable progression conditions without requiring human intervention
21. DIDO-TE selects only permitted Execution Paths
22. DIDO-TE applies the applicable Timeouts
23. DIDO-TE handles detected execution exceptions according to the Acceptance Criteria
24. DIDO-TE reaches the state identified as Test Execution completion without requiring human intervention
25. DIDO-TE records any human control operation performed after initiation
26. DIDO-TE produces the required Test Results
27. The verification record preserves Traceability among the Acceptance Criteria, Test Definition, Test Environment, Test Execution, Test Run, performed Test Steps, Execution Path, Test Results, human control operations, and supporting Evidence

The following pages reference this requirement:

TBD

Assess the current DIDO-TE implementation against the automated Test Execution obligation established by MO-003.

Review and accept MO-003 as a proposed Mission Objective consolidated from the automated Test Execution intent in the DIDO-TE source material.

The following unresolved issues affect this requirement:

Identify the authoritative source requirements, original requirement identifiers, and controlling citations for automated Test Execution.

Define the event constituting initiation of automated Test Execution.

Define the event constituting completion of automated Test Execution.

Define the human activities permitted before Test Execution initiation.

Define the human control operations permitted after Test Execution initiation.

Define whether every Test Step in the selected Test Execution must be automated.

Define the required behavior when the selected Test Definition contains a Test Step requiring human performance.

Define the required behavior when an applicable Test Executable or Execution Facility is unavailable.

Define the exception-handling, retry, pause, cancellation, and recovery behavior applicable to automated Test Execution.

Define the treatment of failed, incomplete, interrupted, timed-out, and canceled Test Runs.

Define the Test Results and Evidence DIDO-TE must produce during automated Test Execution.

This requirement page should retain the stable requirement identifier MO-003.

This page is a leaf requirement page and omits a trailing :start from its namespace.

MO-003 consolidates mission-level intent from multiple source statements. MO-003 does not result from splitting a single parent requirement and therefore uses a Source section rather than a Derived From section.

Changes to the Statement should preserve:

• DIDO-TE as the responsible Actor
• A selected Test Execution as the automated activity
• Test Execution initiation as the beginning of the automation boundary
• Test Execution completion as the required outcome
• Absence of required human intervention between initiation and completion as the automation constraint

The supporting requirements should define:

• Test Execution selection
• Test Definition selection
• Test Environment selection
• Test Execution initiation
• Test Execution completion
• Pre-execution human activities
• Permitted human control operations
• Automated Test Steps
• Test Executables
• Execution Facilities
• Test Argument Values
• Starting Conditions
• Timing Conditions
• Execution Paths
• Timeouts
• Exception handling
• Retry behavior
• Pause behavior
• Cancellation behavior
• Recovery behavior
• Test Result generation
• Evidence generation
• Provenance
• Traceability

A permitted human request to pause, cancel, or otherwise control an automated Test Run should not constitute a failure of MO-003 when the Acceptance Criteria authorize the operation.

A Test Definition containing a Test Step requiring human performance cannot satisfy MO-003 unless the Acceptance Criteria exclude the Test Step from the selected automated Test Execution or an automated Test Executable implements the Test Step.

Material changes should receive review and should update the verification criteria, source records, supporting requirements, and Issues section.

To reference this requirement Statement from another Wiki page, insert:

{{section>dido:03-dido-te:99-annexes:annex-c-requirements:01-mission-objectives:mo-003#Statement&noheader&nofooter&noeditbtn}}

Do not rename this page after an external citation unless a redirect or move plan is in place.