Artificial intelligence (AI) is becoming increasingly common across industries, often without full consideration of ethical implications. Many AI models are now deployed in decision-making contexts, where bias in training data can perpetuate inequities and lead to harmful outcomes. As future engineers, students must understand both the power and risks of AI so they can make responsible decisions about its deployment.
This study assesses the effectiveness of a classroom case study designed to teach the ethical challenges of AI [1]. The case centers on Amazon’s attempt to implement an AI-based résumé screening model. Trained on historical employee data, the model systematically disadvantaged female applicants, ultimately prompting its discontinuation. Using a synthetic dataset and active learning techniques, the case study engages students in identifying and mitigating algorithmic bias.
The case was implemented in two graduate-level courses: an Ethics in Automation class and an Introduction to Machine Learning course. We evaluated learning outcomes using four objectives: (1) recognizing bias in training data, (2) interpreting machine learning model outcomes, (3) evaluating bias mitigation strategies, and (4) reflecting on ethical challenges in AI. Students completed identical pre- and post-class surveys (13 Likert-scale items) along with post-class quantitative feedback.
We collected responses from N = 16 ethics students and N = 21 machine learning students. Across all questions, we observed learning gains. Normalized gains for the questions most closely aligned with each objective were 65.3% for recognizing bias, 50.4% for interpreting results, 45.0% for evaluating mitigation strategies, and 50.5% for ethical reflection. These outcomes suggest that the case study effectively increased student awareness and confidence in addressing the ethical dimensions of AI.
Ethics students showed greater baseline familiarity with bias and ethics (pre-class average 3.38 vs. 2.69), while machine learning students demonstrated higher normalized gains (45% vs. 55%), suggesting strong adaptability despite limited prior exposure.
This case study demonstrates an effective way to engage students with the social impacts of AI while building technical literacy. It has been used in graduate-level ethics and machine learning courses and can be easily adapted for undergraduate audiences or other disciplines. To support broader adoption, we have made the dataset and accompanying Jupyter notebook available on GitHub [2].
[1] Haughey, Annika, Brian P. Mann, and Siobhan Oca. "Case Study: Using Synthetic Datasets to Examine Bias in Machine Learning Algorithms for Resume Screening." 2025 ASEE Annual Conference & Exposition. 2025.
[2] Redacted for Anonymity
Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.