A water-utility agent receives conflicting supervisor prompts and selects the action that skips a safety interlock. What principle failed?
Select an answer to reveal the explanation.
Short Explanation
When two supervisors give you conflicting orders, choosing the one that skips a safety interlock is not autonomy—it is a failure of goal integrity. You need safety rules that resolve conflicts and protect the interlock. Conflicting prompts without those checks can lead to unsafe actions.
Full Explanation
A water-utility agent that receives conflicting supervisor prompts and selects the action that skips a safety interlock has failed goal integrity and safe conflict handling when instructions disagree. In SCADA-adjacent or plant operations, an agent must detect conflicting directives, prioritize safety constraints, and refuse or escalate when an instruction would bypass an interlock. A utility cannot treat “the prompt that wins” as acceptable autonomy when physical safety controls are at stake. Maximizing token usage so every prompt is answered at once fails because throughput does not resolve unsafe conflicts and may encourage the agent to execute contradictory actions. Retaining SCADA credentials in persistent memory for faster outage response increases credential exposure and does not restore safe conflict handling. Disabling telemetry so operators cannot see which prompt won removes accountability and hides the failure rather than correcting goal integrity. Exam caveat: conflict-handling policies must encode non-bypassable safety rules; treating conflicting prompts as simultaneously binding is not a safety design and can still lead to skipped interlocks. Practical check: in a water-operations simulator, provide two conflicting supervisor instructions, including one that would skip an interlock, and verify the agent refuses or escalates to a human, logs the conflict, and never sends the unsafe command to the plant interface.