An AI ethics board is reviewing an AI system that predicts student dropout risk in higher education. Students flagged as high dropout risk receive additional academic support. However, the AI risk team identifies that the model also shares flags with scholarship committees. Which ethical concern is MOST critical?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you're seeing purpose creep when dropout-risk data collected to help students gets shared with scholarship committees. That turns a welfare tool into a potential discrimination tool and violates purpose limitation.
Full Explanation
Sharing dropout risk flags with scholarship committees is the most critical concern because collecting data to support student welfare and then sharing that data for scholarship decisions represents purpose creep and violates the ethical principle of purpose limitation. Dropout risk flags used in scholarship decisions could disadvantage already vulnerable students, compounding inequity rather than addressing it. This also raises FERPA and GDPR compliance concerns. The concern that the model may not be accurate enough to justify the cost of additional academic support is secondary compared to the misuse of the data. The concern that the model should be more transparent about which features drive individual dropout predictions is important but secondary to the misuse. The concern that students should be required to consent to academic support before the model can flag them addresses a different concern.