A data extraction pipeline uses few-shot prompting with 8 examples to extract structured fields from insurance claim forms. The system performs well on training-distribution claims (85% accuracy) but drops to 52% on claims from a newly onboarded insurer with different form layouts. The team's options are: (A) add 8 more examples from the new insurer's forms, (B) restructure the prompt to be layout-agnostic using field descriptions instead of position-based examples, (C) fine-tune a Claude model on the new insurer's data. Which approach is architecturally correct for a production system serving multiple insurers with heterogeneous form layouts?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — the combined architecture of layout-agnostic base prompt plus dynamically selected per-insurer examples is the production-correct solution for a multi-insurer system. Approach B alone (layout-agnostic restructure) improves generalization but loses the specificity that drives high accuracy — field descriptions without examples leave Claude to interpret ambiguous form elements.
Full explanation below image
Full Explanation
The combined architecture of layout-agnostic base prompt plus dynamically selected per-insurer examples is the production-correct solution for a multi-insurer system. Approach B alone (layout-agnostic restructure) improves generalization but loses the specificity that drives high accuracy — field descriptions without examples leave Claude to interpret ambiguous form elements. Approach A alone (add more examples) is a short-term fix that creates a maintenance nightmare: every new insurer requires new examples, example sets grow indefinitely, and prompt length approaches context limits. The optimal architecture: (1) redesign the base prompt with semantic field descriptions, (2) maintain a per-insurer example library, (3) detect the insurer/form type at inference time (from metadata or layout detection), (4) dynamically inject 4-6 relevant examples from that insurer's library. This scales to any number of insurers with O(1) prompt engineering per insurer. Option C (fine-tuning) is tempting but wrong for Claude — Claude models support fine-tuning only in specific contexts, and fine-tuning on structured extraction creates a rigid model that requires retraining when any insurer updates their forms.