dido:03-dido-te:99-annexes:annex-c-requirements:01-mission-objectives:mo-003

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		 Previous revision
dido:03-dido-te:99-annexes:annex-c-requirements:01-mission-objectives:mo-003 [2026/08/06 07:40] – created nick_didodido:03-dido-te:99-annexes:annex-c-requirements:01-mission-objectives:mo-003 [2026/08/06 08:11] (current) – nick_dido
Line 21: Line 21:
 <todo>Identify the authoritative source requirements, original requirement identifiers, and controlling citations for the automated Test Execution objective.</todo> <todo>Identify the authoritative source requirements, original requirement identifiers, and controlling citations for the automated Test Execution objective.</todo>
  
-MO-003 expresses the common mission-level outcome. Detailed obligations addressing Test Execution initiation, sequencing, scheduling, coordination, monitoring, exception handling, termination, Test Result generation, and Evidence preservation remain separate Operational, Functional, Logging and Auditing, Reliability, Performance, and Data Management [[dido:99_annexes:annex-b-terms-and-definitions:r:requirement|Requirements]].+MO-003 expresses the common mission-level outcome. Detailed obligations addressing Test Execution initiation, sequencing, scheduling, coordination, monitoring, [[dido:99_annexes:annex-b-terms-and-definitions:e:exception_handling|Exception Handling]], termination, Test Result generation, and Evidence preservation remain separate Operational, Functional, Logging and Auditing, Reliability, Performance, and Data Management [[dido:99_annexes:annex-b-terms-and-definitions:r:requirement|Requirements]].
  
 ===== Rationale ===== ===== Rationale =====
Line 37: Line 37:
   * Selection of permitted [[dido:99_annexes:annex-b-terms-and-definitions:e:execution_path|Execution Paths]]   * Selection of permitted [[dido:99_annexes:annex-b-terms-and-definitions:e:execution_path|Execution Paths]]
   * Enforcement of applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:timeout|Timeouts]]   * Enforcement of applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:timeout|Timeouts]]
 +  * Detection of [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution_exception|Test Execution Exceptions]]
 +  * Performance of applicable [[dido:99_annexes:annex-b-terms-and-definitions:e:exception_handling|Exception Handling]]
   * Coordination among [[dido:99_annexes:annex-b-terms-and-definitions:n:node|Nodes]] and [[dido:99_annexes:annex-b-terms-and-definitions:n:node_set|Node Sets]]   * Coordination among [[dido:99_annexes:annex-b-terms-and-definitions:n:node|Nodes]] and [[dido:99_annexes:annex-b-terms-and-definitions:n:node_set|Node Sets]]
   * Collection of observations and measurements   * Collection of observations and measurements
Line 59: Line 61:
   * Making a [[dido:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]]   * Making a [[dido:99_annexes:annex-b-terms-and-definitions:v:validation_decision|Validation Decision]]
  
-After initiation, DIDO-TE performs the selected Test Execution without requiring a human Actor to perform a Test Step, select an Execution Path, transfer an intermediate result, or advance the Test Sequence.+After initiation, DIDO-TE performs the selected Test Execution without requiring a human Actor to: 
 + 
 +  * Perform a Test Step 
 +  * Select an Execution Path 
 +  * Transfer an intermediate result 
 +  * Advance the Test Sequence 
 +  * Detect a Test Execution Exception 
 +  * Perform automated Exception Handling 
 +  * Determine whether the Test Run satisfied its completion condition 
 +  * Assign the applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_state|Test Run State]]
  
 An authorized human request to pause, cancel, or otherwise control a Test Run does not by itself demonstrate a failure of automated Test Execution when the applicable Acceptance Criteria permit the requested operation. An authorized human request to pause, cancel, or otherwise control a Test Run does not by itself demonstrate a failure of automated Test Execution when the applicable Acceptance Criteria permit the requested operation.
 +
 +A human notification does not by itself constitute human intervention. A human response affecting the progression of Test Execution constitutes human intervention.
 +
 +Exception Handling requiring an Actor to select a response, authorize continued Test Execution, supply information, or perform an operation does not satisfy the automation constraint during the period in which DIDO-TE requires the Actor’s response.
 +
 +A detected Test Execution Exception does not necessarily prevent completion of automated Test Execution. Automated Exception Handling can resolve the Test Execution Exception and permit the Test Run to continue.
 +
 +When Exception Handling does not resolve the Test Execution Exception, DIDO-TE can place the Test Run in the Terminated Test Run State and record the applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_termination_reason|Test Run Termination Reason]].
 +
 +A Test Run that enters the Terminated Test Run State without satisfying its defined completion condition is an [[dido:99_annexes:annex-b-terms-and-definitions:i:incomplete_test_run|Incomplete Test Run]].
  
 The detailed requirements establish: The detailed requirements establish:
Line 74: Line 95:
   * The required [[dido:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executables]]   * The required [[dido:99_annexes:annex-b-terms-and-definitions:t:test_executable|Test Executables]]
   * The required [[dido:99_annexes:annex-b-terms-and-definitions:e:execution_facility|Execution Facilities]]   * The required [[dido:99_annexes:annex-b-terms-and-definitions:e:execution_facility|Execution Facilities]]
-  * The applicable exception-handling behavior+  * The Test Execution Exceptions subject to detection 
 +  * The applicable Exception Handling 
 +  * The applicable Test Run States 
 +  * The permitted Test Run State transitions 
 +  * The applicable Test Run Termination Reasons 
 +  * The treatment of Incomplete Test Runs
   * The required Test Results and Evidence   * The required Test Results and Evidence
  
Line 102: Line 128:
   * [[dido:99_annexes:annex-b-terms-and-definitions:e:execution_path|Execution Paths]]   * [[dido:99_annexes:annex-b-terms-and-definitions:e:execution_path|Execution Paths]]
   * [[dido:99_annexes:annex-b-terms-and-definitions:t:timeout|Timeouts]]   * [[dido:99_annexes:annex-b-terms-and-definitions:t:timeout|Timeouts]]
 +  * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution_exception|Test Execution Exceptions]]
 +  * [[dido:99_annexes:annex-b-terms-and-definitions:e:exception_handling|Exception Handling]]
   * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environments]]   * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_environment|Test Environments]]
   * [[dido:99_annexes:annex-b-terms-and-definitions:n:node|Nodes]]   * [[dido:99_annexes:annex-b-terms-and-definitions:n:node|Nodes]]
   * [[dido:99_annexes:annex-b-terms-and-definitions:n:node_set|Node Sets]]   * [[dido:99_annexes:annex-b-terms-and-definitions:n:node_set|Node Sets]]
   * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Runs]]   * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run|Test Runs]]
 +  * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_state|Test Run States]]
 +  * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_termination_reason|Test Run Termination Reasons]]
 +  * [[dido:99_annexes:annex-b-terms-and-definitions:i:incomplete_test_run|Incomplete Test Runs]]
   * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]]   * [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]]
-  * Failed Test Runs 
-  * Incomplete Test Runs 
-  * Interrupted Test Runs 
-  * Timed-out Test Runs 
-  * Canceled Test Runs 
   * [[dido:99_annexes:annex-b-terms-and-definitions:c:connected_environment|Connected Environments]]   * [[dido:99_annexes:annex-b-terms-and-definitions:c:connected_environment|Connected Environments]]
   * [[dido:99_annexes:annex-b-terms-and-definitions:d:disconnected_environment|Disconnected Environments]]   * [[dido:99_annexes:annex-b-terms-and-definitions:d:disconnected_environment|Disconnected Environments]]
Line 138: Line 164:
   - The Acceptance Criteria identify the permitted [[dido:99_annexes:annex-b-terms-and-definitions:e:execution_path|Execution Paths]]   - The Acceptance Criteria identify the permitted [[dido:99_annexes:annex-b-terms-and-definitions:e:execution_path|Execution Paths]]
   - The Acceptance Criteria identify the applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:timeout|Timeouts]]   - The Acceptance Criteria identify the applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:timeout|Timeouts]]
 +  - The Acceptance Criteria identify the [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution_exception|Test Execution Exceptions]] subject to detection
 +  - The Acceptance Criteria identify the applicable [[dido:99_annexes:annex-b-terms-and-definitions:e:exception_handling|Exception Handling]]
 +  - The Acceptance Criteria identify any Exception Handling requiring human intervention
 +  - The Acceptance Criteria identify the applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_state|Test Run States]]
 +  - The Acceptance Criteria identify the permitted transitions between Test Run States
 +  - The Acceptance Criteria identify the applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_termination_reason|Test Run Termination Reasons]]
 +  - The Acceptance Criteria identify the treatment of [[dido:99_annexes:annex-b-terms-and-definitions:i:incomplete_test_run|Incomplete Test Runs]] for each applicable Test Run Termination Reason
   - An authorized Actor initiates the selected Test Execution   - An authorized Actor initiates the selected Test Execution
   - DIDO-TE performs each automated Test Step after initiation   - DIDO-TE performs each automated Test Step after initiation
Line 145: Line 178:
   - DIDO-TE selects only permitted Execution Paths   - DIDO-TE selects only permitted Execution Paths
   - DIDO-TE applies the applicable Timeouts   - DIDO-TE applies the applicable Timeouts
-  - DIDO-TE handles detected execution exceptions according to the Acceptance Criteria+  - DIDO-TE detects the Test Execution Exceptions identified by the Acceptance Criteria 
 +  - DIDO-TE performs the applicable Exception Handling for each detected Test Execution Exception 
 +  - DIDO-TE performs automated Exception Handling without requiring human intervention 
 +  - DIDO-TE records each detected Test Execution Exception 
 +  - DIDO-TE records the Exception Handling performed for each detected Test Execution Exception 
 +  - DIDO-TE records each applicable Test Run State 
 +  - DIDO-TE records each transition between Test Run States 
 +  - DIDO-TE assigns the Completed Test Run State when the Test Run satisfies its defined completion condition 
 +  - DIDO-TE assigns the Terminated Test Run State when the Test Run stops without satisfying its defined completion condition 
 +  - DIDO-TE records the applicable Test Run Termination Reason when the Test Run enters the Terminated Test Run State 
 +  - DIDO-TE identifies a Test Run that enters the Terminated Test Run State without satisfying its defined completion condition as an Incomplete Test Run
   - DIDO-TE reaches the state identified as Test Execution completion without requiring human intervention   - DIDO-TE reaches the state identified as Test Execution completion without requiring human intervention
   - DIDO-TE records any human control operation performed after initiation   - DIDO-TE records any human control operation performed after initiation
   - DIDO-TE produces the required [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]]   - DIDO-TE produces the required [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Results]]
-  - The verification record preserves [[dido:99_annexes:annex-b-terms-and-definitions:t:traceability|Traceability]] among the Acceptance Criteria, Test Definition, Test Environment, Test Execution, Test Run, performed Test Steps, Execution Path, Test Results, human control operations, and supporting [[dido:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]]+  - The verification record preserves [[dido:99_annexes:annex-b-terms-and-definitions:t:traceability|Traceability]] among the Acceptance Criteria, Test Definition, Test Environment, Test Execution, Test Run, Test Run State transitions, detected Test Execution Exceptions, performed Exception Handling, applicable Test Run Termination Reason, Incomplete Test Run classification, performed Test Steps, Execution Path, Test Results, human control operations, and supporting [[dido:99_annexes:annex-b-terms-and-definitions:e:evidence|Evidence]]
  
 ===== Referenced By ===== ===== Referenced By =====
Line 190: Line 233:
 <todo>Define the required behavior when an applicable Test Executable or Execution Facility is unavailable.</todo> <todo>Define the required behavior when an applicable Test Executable or Execution Facility is unavailable.</todo>
  
-<todo>Define the exception-handling, retry, pause, cancellation, and recovery behavior applicable to automated Test Execution.</todo>+<todo>Identify the Test Execution Exceptions DIDO-TE must detect during automated Test Execution.</todo>
  
-<todo>Define the treatment of failed, incomplete, interrupted, timed-out, and canceled Test Runs.</todo>+<todo>Define the Exception Handling applicable to each Test Execution Exception.</todo> 
 + 
 +<todo>Define the retry, pause, cancellation, termination, compensation, and recovery operations permitted during automated Exception Handling.</todo> 
 + 
 +<todo>Define which Exception Handling operations can require human intervention.</todo> 
 + 
 +<todo>Define the treatment of Incomplete Test Runs for each applicable Test Run Termination Reason.</todo>
  
 <todo>Define the Test Results and Evidence DIDO-TE must produce during automated Test Execution.</todo> <todo>Define the Test Results and Evidence DIDO-TE must produce during automated Test Execution.</todo>
Line 220: Line 269:
   * Test Execution initiation   * Test Execution initiation
   * Test Execution completion   * Test Execution completion
 +  * Test Run States
 +  * Test Run State transitions
 +  * Test Run Termination Reasons
 +  * Incomplete Test Runs
   * Pre-execution human activities   * Pre-execution human activities
   * Permitted human control operations   * Permitted human control operations
Line 230: Line 283:
   * Execution Paths   * Execution Paths
   * Timeouts   * Timeouts
-  * Exception handling+  * Test Execution Exception detection 
 +  * Exception Handling
   * Retry behavior   * Retry behavior
   * Pause behavior   * Pause behavior
   * Cancellation behavior   * Cancellation behavior
 +  * Termination behavior
 +  * Compensation behavior
   * Recovery behavior   * Recovery behavior
   * Test Result generation   * Test Result generation
Line 243: Line 299:
  
 A Test Definition containing a Test Step requiring human performance cannot satisfy MO-003 unless the Acceptance Criteria exclude the Test Step from the selected automated Test Execution or an automated Test Executable implements the Test Step. A Test Definition containing a Test Step requiring human performance cannot satisfy MO-003 unless the Acceptance Criteria exclude the Test Step from the selected automated Test Execution or an automated Test Executable implements the Test Step.
 +
 +A Test Run that enters the Completed [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_state|Test Run State]] can produce a [[dido:99_annexes:annex-b-terms-and-definitions:t:test_result|Test Result]] assigned a failing [[dido:99_annexes:annex-b-terms-and-definitions:v:verdict|Verdict]]. A failing Verdict does not by itself make the Test Run an [[dido:99_annexes:annex-b-terms-and-definitions:i:incomplete_test_run|Incomplete Test Run]].
 +
 +A Test Run that enters the Terminated Test Run State has an applicable [[dido:99_annexes:annex-b-terms-and-definitions:t:test_run_termination_reason|Test Run Termination Reason]] and is an Incomplete Test Run when it did not satisfy its defined completion condition.
 +
 +A detected [[dido:99_annexes:annex-b-terms-and-definitions:t:test_execution_exception|Test Execution Exception]] does not necessarily prevent DIDO-TE from satisfying MO-003. DIDO-TE can satisfy MO-003 when automated [[dido:99_annexes:annex-b-terms-and-definitions:e:exception_handling|Exception Handling]] resolves the Test Execution Exception and Test Execution reaches its defined completion state without human intervention.
 +
 +Exception Handling requiring an Actor to select a response, authorize continuation, supply information, or perform an operation constitutes human intervention for MO-003.
  
 Material changes should receive review and should update the verification criteria, source records, supporting requirements, and Issues section. Material changes should receive review and should update the verification criteria, source records, supporting requirements, and Issues section.
  • dido/03-dido-te/99-annexes/annex-c-requirements/01-mission-objectives/mo-003.1786027250.txt.gz
  • Last modified: 2026/08/06 07:40
  • by nick_dido