NOD-010f — Recover from a Node Fault

Go to NOD-010 — Inject Node Faults

The DIDO-TE SHALL recover each Node affected by Fault Injection using the governing Test Procedure.

• NOD-010 — Inject Node Faults
• NOD-010a — Specify a Node Fault
• NOD-010b — Authorize Node Fault Injection
• NOD-010c — Prepare Node Fault Injection
• NOD-010d — Inject a Node Fault
• NOD-010e — Control a Node Fault
• [DTE5] DIDO-TE Requirements Register, source requirement identifier and obligation to be assigned
• [DTE6] Structured Information Processing Reference Architecture (SIP-RA)
• [DTE7] Federated Data Interpretation Systems Reference Architecture (FDIS-RA)

Recovery establishes the post-injection condition of the target Node and each Test Environment component affected by the introduced Fault.

The governing Test Definition specifies the recovery conditions and required resulting states. The governing Test Procedure specifies the actions used to terminate or remove the Fault, restore the Node, restore affected Node Bindings and Test Resources, verify resulting states, and disposition unresolved conditions.

Recovery does not presume restoration to the state that existed before Fault Injection. The Test Definition may require a ready, stopped, isolated, degraded, restored, reconfigured, or other identified resulting state.

The DIDO-TE preserves information required to evaluate the Fault and Node response before recovery actions alter the observed conditions.

Incomplete or unverified recovery may leave residual Fault conditions, unauthorized access, altered Node Configuration, invalid Node Bindings, unavailable Test Resources, or hidden effects that contaminate later Test Executions.

• DIDO-TE
• Fault
• Fault Injection
• Node
• Node Configuration
• Node Binding
• Dependency
• Test Definition
• Test Procedure
• Test Environment
• Test Execution
• Test Execution Exception
• Test Resource
• Test Outcome
• Test Result
• Evidence
• Traceability

Verification confirms that:

1. The governing Test Definition specifies each condition that initiates recovery.
2. The governing Test Definition specifies the required resulting states of the Node and affected Test Environment components.
3. The governing Test Procedure specifies the recovery actions and their sequence.
4. The DIDO-TE preserves the required observations before beginning recovery.
5. The DIDO-TE terminates or removes the introduced Fault through the governing Test Procedure.
6. The DIDO-TE verifies that the Fault no longer persists when the Test Definition requires its removal.
7. The DIDO-TE restores or places the Node in the resulting state specified by the governing Test Definition.
8. The DIDO-TE restores, closes, releases, or retains each affected Node Binding and Test Resource as specified.
9. The DIDO-TE identifies each residual effect of the Fault or recovery actions.
10. The DIDO-TE verifies the Node Configuration, lifecycle state, readiness state, isolation state, Node Bindings, Dependencies, and resource conditions after recovery.
11. The DIDO-TE prevents the Node from resuming activity that its resulting state does not permit.
12. The DIDO-TE prevents a recovered Node from participating in another Test Execution before it satisfies the governing readiness and validation conditions.
13. The DIDO-TE detects a missing, incomplete, unsuccessful, interrupted, incorrectly sequenced, or unauthorized recovery action.
14. The DIDO-TE detects an unknown, unexpected, prohibited, or unverifiable resulting state.
15. The DIDO-TE initiates exception handling for each unresolved recovery condition.
16. The DIDO-TE prevents unsuccessful recovery from affecting unrelated Nodes or Test Executions.
17. The DIDO-TE determines the effect of recovery and unresolved conditions on affected Test Executions, Test Outcomes, Test Results, and Evidence.
18. An unsuccessful recovery leaves the Node and affected Test Environment components in known states or identifies their actual states and required corrective action.
19. The DIDO-TE maintains Traceability among the Fault, Fault Injection, authorization, target Node, governing Test Definition, governing Test Procedure, observations, recovery actions, resulting states, exceptions, Test Outcomes, Test Results, and resulting Evidence.
20. Missing, incomplete, unsuccessful, incorrectly sequenced, unauthorized, or untraceable recovery constitutes nonconformance with this requirement.

The following pages reference this requirement:

Assign the delivery phase.

Assign the implementation status.

The proposed derived requirement requires review and acceptance.

Assign the source requirement identifier and obligation from the DIDO-TE Requirements Register.

{{section>dido:03-dido-te:99-annexes:annex-c-requirements:03-node-requirements:nod-010-inject-node-faults:nod-010f-recover-from-a-node-fault#Statement&noheader&nofooter&noeditbtn}}