A water utility MLOps team asks how DoD-style AI ethical principles should shape authorized offensive testing of a plant-operations assistant. What is the soundest application?
Select an answer to reveal the explanation.
Short Explanation
Ethics here aren’t a pass to run prompt injections in live plant systems—they’re the seatbelt. Probe the plant assistant under clear authority, keep humans in the loop, and don’t knock over clean-water ops just for a flashy slide.
Full Explanation
DoD-style AI ethical principles shape authorized offensive testing of a water-utility plant-operations assistant by demanding legality, proportionality, governance, and avoidance of unjustified harm. For public utilities, that means purple-team exercises remain charter-approved, monitored, and designed not to endanger safety-critical processes or resident trust. Ethical framing complements technical methodology: it constrains how tests run (human oversight, stop conditions, non-destructive defaults) rather than banning defensive validation. Running offensive prompt-injection tests directly in live plant systems to ensure realistic reliability assessment ignores proportionality and risks unjustified harm to safety-critical water operations. Relying on traceability logs and model cards to satisfy ethical review while expanding test scope does not constrain how tests run in production and cannot override the need for human oversight and proportionality. Relaxing safety constraints, even inside a sandbox, to isolate and demonstrate an attack pattern still needs explicit written authorization and human oversight; 'sandboxed' framing alone does not satisfy proportionality or governance requirements for a safety-critical water-utility assistant.
Exam caveat: ethics guide lawful, proportional AI testing with oversight—they do not ban testing, nor license unsafe demos. Operational check: bind plant-assistant tests to written RoE with human oversight, prohibit unsafe actuation paths, use non-production or simulated controls where feasible, and abort if clean-water or safety systems could be impacted.