2025 ASEE Annual Conference & Exposition

Experiences with using an LLM-based Chatbot for a Multicultural Engineering Program Orientation (Experience)

Presented at Minorities in Engineering Division(MIND) Technical Session 11

Given the pace with which AI systems are being developed and used and the need for more guidance around the ethical use of AI due to the prominence of artificially intelligent systems, future engineers need to be able to analyze the available AI models and make responsible choices critically. In the Fall of 2024, The Human in Computing and Cognition (THiCC) Lab collaborated with the Multicultural Engineering Program Orientation (MEPO) at Penn State to teach incoming engineering students about the responsible use of AI systems with the help of an interactive Large Language Model (LLM) based chatbot. MEPO is a four-day program designed to welcome incoming first-year undergraduate engineering students primarily from racially and ethnically minoritized groups, fostering connections with upper-division student mentors, academic success resources, and professionals in the field and exposing them to typical elements of the engineering curriculum such as teamwork and innovation. One exciting component of MEPO is an engineering design competition, for which this year, the theme was “decades.” Students were assigned to one of 14 groups. Those groups were then assigned to one of 4 decades. Each decade had an accompanying disaster that the students would be responsible for helping to resolve–the students assigned to the 1910s, for example, were tasked with designing a context-appropriate technological solution to help mitigate the Spanish Flu. The final objective was to create a prototype and a presentation regarding their findings and solutions to their assigned problem. The chatbot was meant to aid the students as they brainstormed different ideas and solutions, allowing them to think critically about these intelligent systems as they used the chatbot. Before the four-day MEPO event, our team at the THiCC lab spent some time building the chatbot for the students to use. We chose the chatbot, LLaMA-2, because of its reliable text generation and open-sourced LLM, which has information on data sources used to train the system. We focused on transparency, as we wanted the students to know where the information came from and to understand the importance of having that knowledge. Additionally, we integrated Retrieval-Augmented Generation (RAG) to allow the chatbot to pull specific information, like historical data and disaster scenarios, from a custom pamphlet prepared by the MEPO team. This ensured the chatbot gave fact-based, relevant answers tied directly to the decades they were working on. The chatbot was optimized to avoid biased or inaccurate responses and was later hosted on Huggingface spaces. On the first day of the MEPO, the THiCC lab team directed a lesson to introduce the students to the chatbot and its utility. The first half of the lesson was spent educating the students on the dangers and considerations of LLMs and AI. The second half was spent showing the students the back end of the chatbot and how to access the Huggingface space. After the lesson, the students could use the chatbot in their design challenge. During the four days, students presented a range of questions and feedback, from technical questions on how to access the chatbot to questions about motives and why they needed to use the chatbot. On the final day of the competition, students presented their designs and were able to thoughtfully consider the chatbot as an imperfect yet valuable tool in their competition.

Authors

Haya Alshayji Pennsylvania State University [biography]

Haya Alshayji is advancing her academic journey as a Ph.D. candidate at Pennsylvania State University's Industrial and Manufacturing Engineering, focusing on integrating machine learning, data science, and statistical modeling in her studies. Recently, she expanded her research experience by joining THiCC Lab as a research assistant. Haya's involvement in these diverse research areas highlights her commitment to leveraging Al for social good and advancing the field of data science.
Deja Workman Pennsylvania State University
Swapnika Dulam Pennsylvania State University
Dr. Lauren A Griggs http://orcid.org/0000-0001-8141-7040 The Pennsylvania State University [biography]

Dr. Lauren Griggs received her B.S. in Engineering Science from The University of Virginia. She received her Doctor of Philosophy in Biomedical Engineering from Virginia Commonwealth University (VCU), where she worked in the field of Cell and Matrix Biomechanics. She completed her Postdoctoral training in the Department of Biomedical Engineering at Vanderbilt University, where she sought to elucidate the mechanical linkage between breast cancer and diabetes. Dr. Griggs joined Penn State in the summer of 2019 as an Assistant Teaching Professor, Director of the Multicultural Engineering Program and Director of the Clark Scholars Program. In this role, Dr. Griggs drives initiatives to improve the recruitment and retention of historically underrepresented groups pursuing degrees in engineering and strives to foster a welcoming environment that celebrates culture and inclusion. She is the advisor for the National Society of Black Engineers and co-advisor for the Society of Hispanic Professional Engineers, and the Multicultural Engineering Graduate Association. Her passion lies in mentoring through meaningful career discussions and helping students gain confidence as well as succeed in their chosen degree fields.
Dixon Zor Pennsylvania State University
Christopher L Dancy The Pennsylvania State University, University Park [biography]

Dr. Chrisopher L. Dancy is an associate professor of Industrial and Manufacturing Engineering, Computer Science and Engineering, and African American Studies at Penn State where he also currently holds the Harold and Inge Marcus Industrial and Manufacturing Career Development professorship. He directs The Human in Computing and Cognition (THiCC) lab, is currently a faculty partner in the Center for Black Digital Research/#DigBlk, and is the incoming co-director of the Center for Black Digital Research/#DigBlk starting summer 2025. He is also an affiliate faculty in the Institute for Computational and Data Sciences, Center for Socially Responsible AI, and Center for Brain, Behavior, and Cognition. His expertise and interests lie at the intersection of human behavior, computational systems, and social structures with a goal of studying the Human in various computing systems and engineering processes. Dr. Dancy is an NSF CAREER award recipient, with a goal of moving AI and computing towards the development of socioculturally competent AI systems, and is co-lead of the Computational Representation of Human Decision Processes research thrust for the NSF funded AI Institute for Societal Decision-Making where he is using computational cognitive systems to study and intervene in the real impacts of racialization and anti-Blackness on decision making processes and outcomes.

Download paper (1.24 MB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session

For those interested in:

computer science
engineering
undergraduate