Domain 1: AI Risk Identification & Assessment

ISACA AI Audit · 49 questions

• An AI risk manager is evaluating a new computer vision model used to detect manufacturing defects. The model achieves 97% overall accuracy, but the risk manager is concerned about performance on rare defect types. Which metric BEST captures risk related to rare defect classes?
• A healthcare AI system predicts patient readmission risk. An audit reveals the model performs significantly worse for patients from low-income zip codes compared to high-income ones, though both groups are represented in the training data. What type of bias is MOST likely present?
• A risk assessment team discovers that an AI model used for employee performance reviews was trained using data from only one regional office, which is predominantly male. What is the PRIMARY risk this introduces?
• An AI audit team is reviewing a natural language processing model used to screen resumes. The team discovers the model assigns lower scores to resumes containing words statistically associated with women's colleges and female-dominated extracurricular activities. This is an example of which risk?
• An AI risk manager is prioritizing which AI models in a large enterprise portfolio require immediate attention. Which combination of factors should drive the HIGHEST risk rating?
• An organization's AI risk framework includes a requirement for 'AI system resilience.' Which risk scenario BEST tests AI system resilience?
• An insurance company's AI model for auto insurance pricing uses telematics data from policyholders' smartphones to calculate risk scores. The model was trained on data from early adopters of the telematics program, who skewed younger, more tech-savvy, and more safety-conscious than the general population. Now the model is applied to all new policyholders. What AI-specific risk does this deployment scenario create?
• An AI risk team is assessing a computer vision system used at airport security to flag prohibited items. The system flags passengers from certain ethnic backgrounds at a rate 2.3 times higher than the overall population, controlling for actual prohibited item carry rates. This finding is BEST described as:
• A compliance officer discovers that the organization's AI hiring tool was trained on application data where the offer/no-offer label was determined by the previous hiring manager, who had documented complaints of gender discrimination. Which data integrity issue is MOST critical?
• An AI model for predicting equipment failure in a nuclear facility is flagged for review because its training data was collected over only six months, during summer operations. The facility also operates during winter with significantly different temperature and load conditions. What risk does this introduce?
• An organization's AI risk team wants to assess the attack surface of an AI system before deployment. Which framework provides the MOST comprehensive vocabulary for cataloguing adversarial threats specific to AI and ML systems?
• An AI system generates personalized pricing for e-commerce customers. Regulatory investigators find that older customers consistently receive higher prices for identical products. The company argues the model uses purchase history and browsing behavior, not age, as inputs. What concept BEST describes the mechanism by which age discrimination could still occur?
• A multinational company uses a single AI model for credit underwriting across 40 countries. An AI risk assessment identifies that the model was trained primarily on North American and Western European data. Which risk is MOST significant for operations in sub-Saharan African markets?
• An AI governance audit finds that an organization's AI risk assessments are performed once at model inception and never updated. What is the MOST significant risk this practice creates?
• An AI system is trained to classify satellite images for urban planning applications. The system incorrectly classifies solar panels as rooftop HVAC units at a high rate. The training dataset contains 50,000 HVAC unit images and 200 solar panel images. Which risk does this illustrate?
• An AI tool used in a criminal justice risk assessment is found to assign substantially different recidivism risk scores to defendants with identical criminal histories when race is inferred from names. Which technical root cause MOST likely explains this finding?
• An AI system is used by a bank to recommend investment products to wealth management clients. During a model review, the team discovers the model consistently recommends higher-fee products to clients with lower financial literacy scores, regardless of suitability. Which risk does this MOST represent?
• An AI model used for clinical trial participant selection achieves excellent overall performance but the audit team discovers it excludes patients with multiple comorbidities at a disproportionately high rate. The model was trained on historically enrolled trial participants, who typically had fewer comorbidities. What AI risk does this BEST illustrate?
• A company uses an AI system to dynamically price ride-sharing services during periods of high demand. During a natural disaster evacuation, the system prices rides at 6x the normal rate, triggering public backlash. Which AI risk category does this MOST represent?
• An AI model used for automated content moderation on a social media platform has a false positive rate for hate speech detection that is three times higher for posts written in African American Vernacular English (AAVE) compared to Standard American English. What type of bias does this illustrate?
• An organization conducts an AI system risk assessment using a heat map that plots likelihood of harm against severity of harm. An AI model for internal employee scheduling scores 'low likelihood, high severity.' How should this risk be managed?
• An AI governance committee is evaluating whether to permit the use of synthetic training data generated by a generative AI model to supplement scarce real training data for a medical diagnosis AI. Which risk should receive the HIGHEST priority in this evaluation?
• An AI system used to screen loan applications for a community development financial institution (CDFI) rejects applications from applicants with thin credit files at a high rate. The CDFI's mission is specifically to serve underbanked populations who typically have thin credit files. What is the MOST appropriate AI risk management response?
• An AI system for automated email phishing detection incorrectly classifies legitimate vendor invoices as phishing attempts at a rate of 12%, causing operational disruptions. The security team defends the high false positive rate as necessary to maintain the system's 99.2% detection rate for actual phishing. How should the AI risk manager evaluate this trade-off?
• A risk analyst discovers that an AI-powered loan approval model produces significantly higher denial rates for applicants from zip codes with large minority populations. Which type of AI risk does this most directly represent?
• During a threat modeling exercise for an AI fraud detection system, the team identifies that an attacker could submit carefully crafted transactions designed to evade detection. What attack category does this represent?
• An AI system used for medical image analysis begins producing incorrect diagnoses six months after deployment despite unchanged inputs. Root cause analysis reveals the model's training data was collected from a different patient population. This scenario best illustrates which risk?
• A bank's AI credit scoring model was trained primarily on data from economically stable periods. Risk managers are concerned about performance during an economic downturn. Which assessment technique is MOST appropriate to evaluate this risk?
• Which of the following BEST describes the risk of model staleness in a deployed machine learning system?
• A threat actor gains access to a publicly available AI model's API and uses it to determine whether specific individuals' records were used in training. Which attack does this describe?
• An organization deploys an AI system to screen job applications. A risk assessment identifies that the model assigns lower scores to resumes containing terms associated with women's colleges. Which risk category does this represent?
• During a risk identification workshop for a new AI deployment, a risk manager proposes using a bow-tie analysis. What is the PRIMARY advantage of this technique for AI risk?
• A data scientist notices that a sentiment analysis model trained on product reviews performs well in testing but poorly on customer service transcripts. Which AI risk concept BEST explains this failure?
• An AI system used for public benefits eligibility determination denies benefits to 15% more applicants from rural areas compared to urban areas, despite similar economic profiles. The model was trained exclusively on urban data. What governance failure does this MOST directly represent?
• A risk manager reviewing an AI-powered insurance underwriting system notices that the model assigns lower premiums to customers whose web browsing activity suggests interest in healthy lifestyle content. Which risk does this MOST directly raise?
• An attacker with access to a company's AI model API submits thousands of carefully chosen queries to reconstruct the model's decision boundary and build a functional replica. Which attack type is this?
• A predictive maintenance AI system at a manufacturing facility fails to flag an imminent equipment failure that caused a production shutdown. Post-incident analysis finds the failure mode was underrepresented in training data. How should this risk be classified?
• A risk assessment team is evaluating an AI system that uses facial recognition to grant physical access to secure facilities. Beyond accuracy, which risk category deserves the MOST emphasis in the risk assessment for this use case?
• An AI risk analyst is preparing a risk register for a content recommendation system. The analyst identifies that the algorithm may create filter bubbles by repeatedly showing users content similar to what they have previously engaged with. How should this risk be categorized?
• An organization is developing an AI risk taxonomy. Which classification BEST organizes AI risks for comprehensive enterprise risk management integration?
• A risk committee is evaluating whether to deploy an AI system for automated debt collection communications. The system was trained primarily on historical communications from accounts that were successfully resolved. What dataset bias risk does this create?
• A retail bank's AI model for mortgage origination has been in production for 14 months. The risk team runs a quarterly performance review and notices the model's Gini coefficient has declined from 0.72 to 0.61. What action is MOST appropriate?
• An AI governance team is conducting a risk assessment of an AI system that scores job applicants for a technology company. Historical hiring data used for training reflects a decade of predominantly male hiring in technical roles. Which risk does this create beyond ordinary model error?
• A risk manager is evaluating an AI system that assigns individual risk scores to millions of customers daily. The system uses an ensemble of 47 models. Which risk does this level of complexity introduce that a simpler model would not?
• An organization's AI risk assessment identifies that a sentiment analysis model produces significantly different results when input text contains non-standard English dialects or code-switching. How should this risk be classified?
• During an AI risk assessment of a clinical decision support system, risk analysts debate whether the AI's recommendation errors are more harmful than equivalent errors by human clinicians. Which risk concept does this debate MOST engage?
• An adversary injects subtle, imperceptible perturbations into medical scan images before submitting them to a diagnostic AI system, causing the system to misclassify cancerous findings as benign. Which attack type does this represent?
• A risk manager identifies that an AI model used for real-time trading decisions has a 200-millisecond inference latency that occasionally spikes to 2,000 milliseconds under load. In this context, what risk does latency variability create?
• An AI risk practitioner is assessing the risks of deploying an AI summarization tool for legal documents. The tool occasionally omits material terms from contract summaries. Which risk category MOST specifically captures the downstream business consequence of this failure?