A medical imaging assistant uses Claude to analyze radiology reports alongside the corresponding DICOM images converted to JPEG. A radiologist notes that the model sometimes contradicts the written report when describing findings visible in the image. What is the most likely cause and correct mitigation?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — c is correct because multimodal models process visual and textual inputs through separate encoding pathways that are fused at attention layers — when image features and text features carry conflicting information, the model's output depends on which modality receives higher attention weight in context, which can vary by phrasing and visual salience. The correct mitigation is prompt-level instruction: explicitly ask the model to identify and flag conflicts between modalities rather than silently resolving them.
Full explanation below image
Full Explanation
C is correct because multimodal models process visual and textual inputs through separate encoding pathways that are fused at attention layers — when image features and text features carry conflicting information, the model's output depends on which modality receives higher attention weight in context, which can vary by phrasing and visual salience. The correct mitigation is prompt-level instruction: explicitly ask the model to identify and flag conflicts between modalities rather than silently resolving them. A is wrong because disabling image inputs eliminates the clinical value of the system — the whole point is to combine image and report analysis. B is wrong because the contradiction is not a domain knowledge problem but a multimodal fusion problem; fine-tuning on more radiology data does not change the fundamental fusion architecture. D is wrong because while JPEG compression affects image quality, the issue described (contradicting report text) reflects a grounding conflict, not compression artifacts obscuring findings.