A data extraction pipeline uses few-shot prompting with 8 examples to extract structured fields from insurance claim forms. The system performs well on training-distribution claims (85% accuracy) but drops to 52% on claims from a newly onboarded insurer with different form layouts. The team's options are: (A) add 8 more examples from the new insurer's forms, (B) restructure the prompt to be layout-agnostic using field descriptions instead of position-based examples, (C) fine-tune a Claude model on the new insurer's data. Which approach is architecturally correct for a production system serving multiple insurers with heterogeneous form layouts?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you need the combined architecture: a layout-agnostic base prompt plus per-insurer examples selected at inference time. The layout-agnostic prompt alone loses the specificity that drives accuracy, and adding examples alone doesn't scale across insurers.
Full Explanation
The production-correct solution is the combined architecture: a layout-agnostic base prompt using semantic field descriptions, combined with per-insurer few-shot examples dynamically selected at inference time based on detected form layout. The layout-agnostic restructure alone improves generalization but loses the specificity that drives high accuracy; field descriptions without examples leave Claude to interpret ambiguous form elements. The add-more-examples approach alone is a short-term fix that creates a maintenance nightmare: every new insurer requires new examples, example sets grow indefinitely, and prompt length approaches context limits. The optimal architecture is to redesign the base prompt with semantic field descriptions, maintain a per-insurer example library, detect the insurer or form type at inference time from metadata or layout detection, and dynamically inject 4-6 relevant examples from that insurer's library. This scales to any number of insurers with constant prompt-engineering effort per insurer. The fine-tuning approach is tempting but wrong for Claude here: Claude models support fine-tuning only in specific contexts, and fine-tuning on structured extraction creates a rigid model that requires retraining when any insurer updates their forms.