Prompt Engineering & Structured Output

CCAR-F · 60 questions

• A city PR bot should flag comments only when claimed behavior contradicts the code. Which prompt approach improves precision?
• DevOps adds only-report-high-confidence wording, yet style nits still flood civic repos. What does that show?
• High false-positive style findings drown out real security issues in city DevOps review. What is the main risk?
• The permitting team wants clearer PR-review prompts. Which design best reduces noise?
• One PR-review category floods the clerk portal team with false positives while its prompt is being rewritten. What should they do meantime?
• Fee-logic review severity keeps drifting between reviewers. What prompt design stabilizes classification?
• Permit validators still over-flag after adding a vague be-conservative instruction. Why?
• Ambiguous 311 routing improves after adding a few worked examples showing why one tool was chosen. What technique is that?
• PR review comments for the water-billing service keep changing shape. How can few-shots help?
• Municipal code review false positives drop after examples contrast acceptable city patterns with real defects. What is being taught?
• Permit PDF extraction struggles when citations are inline in some packets and bibliographic in others. What helps?
• Public-health form extraction leaves empty fields on informal measurements and mixed table/narrative layouts. What should be added?
• Detailed instructions for clerk filing extracts still produce inconsistent fields until examples are added. What does that illustrate?
• After adding few-shots, the parcel-form extractor handles novel layouts it never saw verbatim. What capability is that?
• A clerk filing extractor must return schema-compliant JSON reliably. Which approach is preferred?
• With toolchoice set to auto, Claude sometimes chats instead of extracting permit fields. What should the team do for mandatory extraction?
• Required schema fields for optional permit attributes cause fabricated values. How should the schema change?
• Violation categories for code enforcement keep expanding beyond a fixed list. Which enum pattern helps?
• Mixed document batches for the assessor's office need metadata before enrichment. Which toolchoice pattern fits?
• Strict JSON schemas stop malformed fee extracts, yet stated totals still disagree with line items. What limitation remains?
• Scanned utility bills arrive with messy dates and amounts. What should sit beside a strict extraction schema?
• A parcel extract fails schema checks. How should the retry be framed for correction?
• Parcel retries keep failing because the needed grantor name exists only in an external deed not in the packet. What should the architect conclude?
• Code-review false positives keep recurring for the water-billing service. What structured field helps analyze dismissals?
• Fee extracts pass schema checks but stated totals may disagree with line items. What self-correction pattern helps?
• tooluse already blocks invalid JSON shapes, yet an APN lands in the wrong field. What kind of error is that?
• Multi-page permit packets sometimes list conflicting square footage. What structured signal should the extract include?
• Some extract failures are format mismatches; others lack any source for a required field. How should handling differ?
• Overnight permit-packet extraction is latency-tolerant and cost-sensitive. Which API approach fits?
• Pre-merge CI review for the permitting service must finish before merge gates. Why avoid Message Batches there?
• A batch job for assessor PDFs needs mid-request live MCP parcel lookups. What limitation applies?
• Some oversized assessor packets fail in a batch after chunking. How should failures be recovered?
• Thousands of assessor PDFs are queued for extraction. What should happen before the full batch?
• Batches may take up to about 24 hours, but the clerk SLA is 30 hours end-to-end. What must submission cadence account for?
• Generated CI patches for fee logic need a second look. Which review design is stronger?
• A large multi-file civic PR overwhelms a single review pass. What architecture helps?
• Verification passes for permit extracts need to route human review wisely. What should accompany findings?
• Self-review in the fee-logic generator session keeps defending its own choices. What limitation is showing?
• Cross-file data-flow bugs in a housing-authority PR are missed by single-file comments. Where should that analysis live?
• Subtle authorization bugs in a clerk portal patch were missed by the generating session. What often works better?
• An emergency-management after-action report is drafted by one agent, and leadership is concerned that a single pass may miss factual inconsistencies against the incident logs. What pattern improves reliability here?
• A housing-authority casework review prompt originally said "extract the fields accurately." Extraction quality stayed inconsistent across caseworkers' scanned intake forms. Replacing that line with an explicit, itemized rubric -- e.g., "a household-size field is correct only if it matches a number appearing on the form; do not infer from unit type" -- produced measurably fewer disputed extractions. What does this demonstrate?
• A building-inspection assistant must review several code sections, cross-reference measurements, and only then emit a single structured pass/fail verdict as JSON. The team has Claude reason through the cross-references in an unstructured scratchpad section of its response, then emit the final verdict in a clearly delimited JSON block at the end. What is the benefit of separating scratchpad reasoning from the final structured answer this way?
• A parks-and-recreation facility booking form has a variable number of attendees per reservation, each with a name and an age bracket. Which schema design best represents this in the structured extraction output?
• A housing authority's eligibility team sends Claude a 40-page tenant handbook, three policy memos, and a caseworker's free-text notes in a single prompt, followed by instructions on how to summarize the applicant's situation. The model keeps treating sentences from the caseworker's notes as if they were commands, and occasionally answers a rhetorical question that appears inside a policy memo. Which prompt-construction change most directly fixes this?
• A parks department builds a small service that turns shelter-reservation requests into a JSON record for its booking system. The prompt asks for JSON only, but roughly one response in six begins with a line such as 'Here is the JSON you requested:' before the object, which breaks the downstream parser. Which technique most directly guarantees the response begins as JSON?
• A court clerk's office calls Claude to produce a single tagged block of case metadata and wants generation to halt as soon as the closing tag is emitted, so no trailing commentary is billed or returned. Which API parameter is designed for this?
• An elections office runs a nightly job that extracts the precinct number, ballot style, and machine serial from scanned poll-worker incident forms. Two runs over the identical batch produced records that differed in wording and, on a few forms, in which serial was picked. Which sampling setting best matches this workload?
• A water utility's outage assistant repeats the same 300-word block — its persona, tone rules, and escalation policy — at the top of every user message in a long conversation. Where does this content belong in the Messages API?
• A public-health team asks Claude to answer questions about a 60-page communicable-disease reporting manual. The current prompt states the question first, then pastes the manual after it. Answers are serviceable but frequently miss details that are plainly in the text. What ordering change should the team make first?
• A library system uses Claude to decide whether a donated book fits its collection policy, returning a JSON verdict plus a rationale. A reviewer notices the model often commits to 'accept' in the first field and then writes a rationale that partly argues against it. How should the output be restructured?
• A sanitation department's missed-pickup triage returns a JSON object that its dispatch system parses with a strict reader. The model reliably produces the six agreed fields but sometimes adds helpful extras such as 'confidencenote' or 'suggestedcrew', and the reader throws on unrecognized keys. Which schema change addresses this?
• A code-enforcement office classifies inspection findings into a violation category that feeds monthly reporting. With a free-text category field, the same condition arrives as 'Overgrown vegetation', 'tall grass', and 'weeds/overgrowth', so counts by category are unusable. What is the right fix in the output contract?
• A fire-dispatch quality-review pipeline extracts a structured incident record from call transcripts. When the transcript never states the caller's callback number, the model simply leaves the callbacknumber key out, and downstream code cannot distinguish 'not stated on the call' from 'the extractor failed'. What schema change resolves the ambiguity?
• A county IT group defines a structured-output schema for asset-retirement records that three separate applications will consume. Today the meaning of ambiguous fields such as dispositiondate is explained in a paragraph inside the prompt, and one team has already copied the schema into a new service without that paragraph. What practice should the group adopt?
• A school district's transportation office gets a structured output with five levels of nesting — route, then stops, then per-stop riders, then per-rider accommodations, then contact records. The model fills the top two levels reliably but the deeper objects come back with fields at inconsistent depths and occasional empty containers. Which change most likely restores reliable output?
• A city treasury office extracts payment amounts from vendor remittance letters into a structured record. With the amount declared as a string, the model returns values such as '$12,400.00', 'USD 12400', and '12,400', and the ledger import fails or misreads them. What is the correct schema-level fix?
• An emergency-management office's public-alert drafter has accumulated a prompt with eleven prohibitions: do not speculate, do not use jargon, do not exceed two paragraphs, do not mention road names, and so on. Output quality has gotten worse, not better, and the model sometimes produces text that is technically compliant but useless. What should the office do?
• A GIS team extracts parcel attributes from scanned deed records. When a deed does not state the lot dimensions, the model produces dimensions that are plausible for the neighborhood, and reviewers have caught several fabricated values already. Which prompt-and-schema instruction addresses this most directly?
• A procurement office's bid-summary prompt returns inconsistent formatting: some responses are markdown bullet lists and some are JSON. Reading the prompt end to end, an architect finds an early section that says 'summarize each bid as a short bulleted list' and a later section that says 'return a JSON object with one entry per bid'. What is the correct diagnosis and fix?