MO-003 — Automated Test Execution

Go to C.1 Mission Objectives

DIDO-TE SHALL complete a selected Test Execution without human intervention after initiation.

This requirement consolidates mission-level intent from the DIDO-TE source material addressing:

• Automated execution of testing scenarios
• Automated performance of Test Sequences
• Coordinated performance of testing activities across Nodes
• Delivery of recorded inputs to Nodes
• Collection of outputs from Nodes
• Automated generation of Test Results
• Preservation of supporting Evidence

Identify the authoritative source requirements, original requirement identifiers, and controlling citations for the automated Test Execution objective.

MO-003 expresses the common mission-level outcome. Detailed obligations addressing Test Execution initiation, sequencing, scheduling, coordination, monitoring, Exception Handling, termination, Test Result generation, and Evidence preservation remain separate Operational, Functional, Logging and Auditing, Reliability, Performance, and Data Management Requirements.

Automated Test Execution permits organizations to perform complex and repeated testing consistently without requiring a human Actor to perform each Test Step.

Automation supports:

• Consistent performance of Test Definitions
• Coordinated performance of Test Sequences
• Performance of Test Steps in the defined order
• Application of Test Argument Values
• Evaluation of applicable Starting Conditions
• Evaluation of applicable Timing Conditions
• Selection of permitted Execution Paths
• Enforcement of applicable Timeouts
• Detection of Test Execution Exceptions
• Performance of applicable Exception Handling
• Coordination among Nodes and Node Sets
• Collection of observations and measurements
• Generation of Test Results
• Preservation of Evidence
• Repetition of testing for regression analysis
• Performance of testing in connected, disconnected, and air-gapped environments

The phrase without human intervention after initiation establishes the automation boundary for MO-003.

MO-003 does not prohibit a human Actor from:

• Creating or selecting a Test Definition
• Selecting a Test Environment
• Supplying or approving Test Argument Values
• Establishing Acceptance Criteria
• Authorizing Test Execution
• Initiating a Test Run
• Monitoring Test Execution
• Requesting cancellation
• Reviewing Test Results
• Making a Validation Decision

After initiation, DIDO-TE performs the selected Test Execution without requiring a human Actor to:

• Perform a Test Step
• Select an Execution Path
• Transfer an intermediate result
• Advance the Test Sequence
• Detect a Test Execution Exception
• Perform automated Exception Handling
• Determine whether the Test Run satisfied its completion condition
• Assign the applicable Test Run State

An authorized human request to pause, cancel, or otherwise control a Test Run does not by itself demonstrate a failure of automated Test Execution when the applicable Acceptance Criteria permit the requested operation.

A human notification does not by itself constitute human intervention. A human response affecting the progression of Test Execution constitutes human intervention.

Exception Handling requiring an Actor to select a response, authorize continued Test Execution, supply information, or perform an operation does not satisfy the automation constraint during the period in which DIDO-TE requires the Actor’s response.

A detected Test Execution Exception does not necessarily prevent completion of automated Test Execution. Automated Exception Handling can resolve the Test Execution Exception and permit the Test Run to continue.

When Exception Handling does not resolve the Test Execution Exception, DIDO-TE can place the Test Run in the Terminated Test Run State and record the applicable Test Run Termination Reason.

A Test Run that enters the Terminated Test Run State without satisfying its defined completion condition is an Incomplete Test Run.

The detailed requirements establish:

• The event constituting initiation
• The event constituting completion
• The permitted pre-execution human activities
• The permitted human control operations after initiation
• The applicable Test Definition
• The applicable Test Environment
• The automated Test Steps
• The required Test Executables
• The required Execution Facilities
• The Test Execution Exceptions subject to detection
• The applicable Exception Handling
• The applicable Test Run States
• The permitted Test Run State transitions
• The applicable Test Run Termination Reasons
• The treatment of Incomplete Test Runs
• The required Test Results and Evidence

This requirement applies to:

• DIDO-TE
• Selected Test Execution
• Automated Test Execution
• Acceptance Criteria
• Actors
• Human Actors
• Users
• Test Definitions
• Test Sequences
• Sequence Steps
• Test Steps
• Test Executables
• Executable Instructions
• Executable Artifacts
• Execution Facilities
• Test Arguments
• Test Argument Values
• Starting Conditions
• Timing Conditions
• Execution Paths
• Timeouts
• Test Execution Exceptions
• Exception Handling
• Test Environments
• Nodes
• Node Sets
• Test Runs
• Test Run States
• Test Run Termination Reasons
• Incomplete Test Runs
• Test Results
• Connected Environments
• Disconnected Environments
• Air-Gapped Environments
• Evidence
• Provenance
• Traceability

Verification confirms that:

1. The selected Test Execution is identified
2. The applicable Test Definition and Version are identified
3. The selected Test Environment is identified
4. The Acceptance Criteria identify the event constituting Test Execution initiation
5. The Acceptance Criteria identify the event constituting Test Execution completion
6. The Acceptance Criteria identify the human activities permitted before initiation
7. The Acceptance Criteria identify the human control operations permitted after initiation
8. The Acceptance Criteria identify the Test Steps subject to automated performance
9. The Acceptance Criteria identify the applicable Test Executables
10. The Acceptance Criteria identify the applicable Execution Facilities
11. The Acceptance Criteria identify the applicable Test Argument Values
12. The Acceptance Criteria identify the applicable Starting Conditions
13. The Acceptance Criteria identify the applicable Timing Conditions
14. The Acceptance Criteria identify the permitted Execution Paths
15. The Acceptance Criteria identify the applicable Timeouts
16. The Acceptance Criteria identify the Test Execution Exceptions subject to detection
17. The Acceptance Criteria identify the applicable Exception Handling
18. The Acceptance Criteria identify any Exception Handling requiring human intervention
19. The Acceptance Criteria identify the applicable Test Run States
20. The Acceptance Criteria identify the permitted transitions between Test Run States
21. The Acceptance Criteria identify the applicable Test Run Termination Reasons
22. The Acceptance Criteria identify the treatment of Incomplete Test Runs for each applicable Test Run Termination Reason
23. An authorized Actor initiates the selected Test Execution
24. DIDO-TE performs each automated Test Step after initiation
25. DIDO-TE advances the applicable Test Sequence without requiring human intervention
26. DIDO-TE transfers required intermediate information without requiring human intervention
27. DIDO-TE evaluates applicable progression conditions without requiring human intervention
28. DIDO-TE selects only permitted Execution Paths
29. DIDO-TE applies the applicable Timeouts
30. DIDO-TE detects the Test Execution Exceptions identified by the Acceptance Criteria
31. DIDO-TE performs the applicable Exception Handling for each detected Test Execution Exception
32. DIDO-TE performs automated Exception Handling without requiring human intervention
33. DIDO-TE records each detected Test Execution Exception
34. DIDO-TE records the Exception Handling performed for each detected Test Execution Exception
35. DIDO-TE records each applicable Test Run State
36. DIDO-TE records each transition between Test Run States
37. DIDO-TE assigns the Completed Test Run State when the Test Run satisfies its defined completion condition
38. DIDO-TE assigns the Terminated Test Run State when the Test Run stops without satisfying its defined completion condition
39. DIDO-TE records the applicable Test Run Termination Reason when the Test Run enters the Terminated Test Run State
40. DIDO-TE identifies a Test Run that enters the Terminated Test Run State without satisfying its defined completion condition as an Incomplete Test Run
41. DIDO-TE reaches the state identified as Test Execution completion without requiring human intervention
42. DIDO-TE records any human control operation performed after initiation
43. DIDO-TE produces the required Test Results
44. The verification record preserves Traceability among the Acceptance Criteria, Test Definition, Test Environment, Test Execution, Test Run, Test Run State transitions, detected Test Execution Exceptions, performed Exception Handling, applicable Test Run Termination Reason, Incomplete Test Run classification, performed Test Steps, Execution Path, Test Results, human control operations, and supporting Evidence

The following pages reference this requirement:

TBD

Assess the current DIDO-TE implementation against the automated Test Execution obligation established by MO-003.

Review and accept MO-003 as a proposed Mission Objective consolidated from the automated Test Execution intent in the DIDO-TE source material.

The following unresolved issues affect this requirement:

Identify the authoritative source requirements, original requirement identifiers, and controlling citations for automated Test Execution.

Define the event constituting initiation of automated Test Execution.

Define the event constituting completion of automated Test Execution.

Define the human activities permitted before Test Execution initiation.

Define the human control operations permitted after Test Execution initiation.

Define whether every Test Step in the selected Test Execution must be automated.

Define the required behavior when the selected Test Definition contains a Test Step requiring human performance.

Define the required behavior when an applicable Test Executable or Execution Facility is unavailable.

Identify the Test Execution Exceptions DIDO-TE must detect during automated Test Execution.

Define the Exception Handling applicable to each Test Execution Exception.

Define the retry, pause, cancellation, termination, compensation, and recovery operations permitted during automated Exception Handling.

Define which Exception Handling operations can require human intervention.

Define the treatment of Incomplete Test Runs for each applicable Test Run Termination Reason.

Define the Test Results and Evidence DIDO-TE must produce during automated Test Execution.

This requirement page should retain the stable requirement identifier MO-003.

This page is a leaf requirement page and omits a trailing :start from its namespace.

MO-003 consolidates mission-level intent from multiple source statements. MO-003 does not result from splitting a single parent requirement and therefore uses a Source section rather than a Derived From section.

Changes to the Statement should preserve:

• DIDO-TE as the responsible Actor
• A selected Test Execution as the automated activity
• Test Execution initiation as the beginning of the automation boundary
• Test Execution completion as the required outcome
• Absence of required human intervention between initiation and completion as the automation constraint

The supporting requirements should define:

• Test Execution selection
• Test Definition selection
• Test Environment selection
• Test Execution initiation
• Test Execution completion
• Test Run States
• Test Run State transitions
• Test Run Termination Reasons
• Incomplete Test Runs
• Pre-execution human activities
• Permitted human control operations
• Automated Test Steps
• Test Executables
• Execution Facilities
• Test Argument Values
• Starting Conditions
• Timing Conditions
• Execution Paths
• Timeouts
• Test Execution Exception detection
• Exception Handling
• Retry behavior
• Pause behavior
• Cancellation behavior
• Termination behavior
• Compensation behavior
• Recovery behavior
• Test Result generation
• Evidence generation
• Provenance
• Traceability

A permitted human request to pause, cancel, or otherwise control an automated Test Run should not constitute a failure of MO-003 when the Acceptance Criteria authorize the operation.

A Test Definition containing a Test Step requiring human performance cannot satisfy MO-003 unless the Acceptance Criteria exclude the Test Step from the selected automated Test Execution or an automated Test Executable implements the Test Step.

A Test Run that enters the Completed Test Run State can produce a Test Result assigned a failing Verdict. A failing Verdict does not by itself make the Test Run an Incomplete Test Run.

A Test Run that enters the Terminated Test Run State has an applicable Test Run Termination Reason and is an Incomplete Test Run when it did not satisfy its defined completion condition.

A detected Test Execution Exception does not necessarily prevent DIDO-TE from satisfying MO-003. DIDO-TE can satisfy MO-003 when automated Exception Handling resolves the Test Execution Exception and Test Execution reaches its defined completion state without human intervention.

Exception Handling requiring an Actor to select a response, authorize continuation, supply information, or perform an operation constitutes human intervention for MO-003.

Material changes should receive review and should update the verification criteria, source records, supporting requirements, and Issues section.

To reference this requirement Statement from another Wiki page, insert:

{{section>dido:03-dido-te:99-annexes:annex-c-requirements:01-mission-objectives:mo-003#Statement&noheader&nofooter&noeditbtn}}

Do not rename this page after an external citation unless a redirect or move plan is in place.