An AI risk team is reviewing a third-party vendor's AI model. The vendor provides performance metrics but refuses to share training data, model architecture, or validation methodology. How should the team assess this situation?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because when a vendor's model is a black box, independent third-party audits of the vendor's practices and contractual performance reporting requirements are the most practical mechanisms to obtain assurance without requiring full proprietary disclosure. This balances vendor IP protection with organizational due diligence obligations.
Full explanation below image
Full Explanation
B is correct because when a vendor's model is a black box, independent third-party audits of the vendor's practices and contractual performance reporting requirements are the most practical mechanisms to obtain assurance without requiring full proprietary disclosure. This balances vendor IP protection with organizational due diligence obligations. Accepting metrics at face value (A) is insufficient due diligence. Rejecting all black-box vendors (C) may be impractical and overly restrictive. Post-deployment monitoring (D) is compensating but does not fulfill pre-deployment due diligence.