A team reports that their security-scanning agent produces results that downstream systems cannot reliably parse. On some runs the agent returns a JSON array of vulnerabilities; on other runs it returns a narrative paragraph; on others it returns a partially structured object with inconsistent field names. Automated ticketing that depends on the agent's output fails unpredictably. What is the most appropriate fix?
Select an answer to reveal the explanation.
Short Explanation
If your agent sends JSON one run, a paragraph the next, and a half-structured object after that, downstream systems need a contract. Define a strict output schema and validate every response before it leaves the agent. That gives ticketing a predictable format to parse.
Full Explanation
Defining outputs is a symmetric counterpart to defining inputs, and it is equally important in agent architecture. An agent's output contract — the schema that specifies the structure, field names, data types, and required fields of every response — allows downstream systems to integrate reliably. Without it, outputs vary based on how the model interprets its instructions on a given run, producing the kind of inconsistency described in this scenario.
The correct fix is the structured output schema option. Define a strict JSON output schema, for example using JSON Schema or Pydantic models, and enforce it at the agent boundary. The schema should specify an array field named vulnerabilities, each item containing id, severity with an enum of critical, high, medium, or low, description as a string, affected_file as a string, and line_number as an integer. The agent must be instructed, via system prompt and function calling with a required output structure, to always return this format. An output validation step should verify schema compliance before results are sent downstream. Non-compliant outputs should be caught and either regenerated or flagged for review.
The option to increase the temperature setting on the agent's model calls to make outputs more deterministic is wrong. Increasing temperature generally increases randomness rather than making outputs more deterministic. Even if temperature is lowered, that only reduces variability in token selection; it does not guarantee structured output. Temperature controls creativity, not schema adherence, and a model without an enforced output format can still vary its structure.
The option to switch the agent from an LLM-based approach to a static regex-based vulnerability scanner is wrong. That is an architectural overreaction. AI-based scanners often detect semantic vulnerabilities that regexes cannot. The problem is output consistency, not detection quality. A well-configured AI agent with an enforced schema is both powerful and consistent.
The option to add error handling in the downstream ticketing system to accommodate all three output formats is wrong. It treats the symptom rather than the disease. It adds fragile format-detection logic in a place that should not need it, increases maintenance burden, and still fails when a fourth unexpected format appears.
For the exam: 'inconsistent outputs,' 'downstream parsing failures,' 'different formats on each run,' and 'unpredictable structure' all point to a missing or unenforced output schema definition.