2025 ASEE Annual Conference & Exposition

Student, Faculty, and Industry Perceptions about Generative AI’s Capabilities to Perform Specific Tasks

As Generative Artificial Intelligence (GAI) tools such as ChatGPT and Copilot become more prevalent in the classroom, workplace, and beyond, there is an urgent need to develop appropriate GAI literacy. One existing challenge in the area of GAI literacy is knowing if and how AI should be used for various tasks, and a key element of the relevant decision-making process is understanding the current capabilities and limitations of GAI tools.

This work explores perceptions about GAI’s capabilities within three groups: engineering students in an upper-division machine learning course, faculty from various disciplines in a GAI professional development course, and industry representatives supporting project-based engineering. Each of these groups participated in an activity consisting of two parts: 1) participants individually rating their confidence that ChatGPT would be able to complete specific tasks (e.g. giving them citations for a research paper, taking an integral, etc.), and 2) participants working in groups to test how well ChatGPT was actually able to complete the given tasks. Data from each of these components were analyzed to better understand how students and faculty perceive the capabilities of GAI and what approaches they take when testing these tools.

This study provides value through its methods, results, and analysis. The methods of the study offer a classroom or professional development activity that can be completed with a variety of audiences to help participants better understand the capabilities and limitations of GAI. The results of the study provide insight into how student and faculty perceptions compare to current performance of tools like ChatGPT and Copilot. Finally, the analysis of the study offers takeaways and suggestions for engineering educators about the implications of these results in the context of teaching and learning in the era of AI technologies.

Authors

Dr. Lauren Singelmann http://orcid.org/https://0000-0003-3586-4266 Minnesota State University, Mankato [biography]

Lauren Singelmann is an assistant professor at Minnesota State University, Mankato through Iron Range Engineering. She earned her PhD in Electrical and Computer Engineering and STEM Education.
Dr. Jack Elliott http://orcid.org/https://0009-0006-5833-0557 Minnesota State University, Mankato [biography]

Jack Elliott is an assistant professor at Iron Range Engineering, a part of Minnesota State University Mankato. Dr. Elliott received his PhD in Engineering Education and M.S. in Mechanical Engineering from Utah State University as an NSF Graduate Research Fellow. His research includes student social support networks in engineering education, experimental fluid dynamics, and developing low-cost technology-based tools for improving fluid dynamics education.
Dr. Yuezhou Wang http://orcid.org/0000-0003-1614-9590 Minnesota State University, Mankato [biography]

Dr. Yuezhou Wang is an associate professor in both Iron Range Engineering and Twin Cities Engineering programs. He received his B.S. in Mechanical Engineering from Shanghai Jiaotong University, China (2008) and Ph.D. in Materials Science and Engineering from University of Minnesota, Twin Cities (2017). His leading teaching competencies are in areas of materials science, structural analysis, finite element modeling and dynamic systems. He has a broad range of research interests. His technical research focuses on multiscale modeling on mechanical behavior of nanofibers and carbon nanotube materials. In the area of pedagogical research, he is interested in using learning analytics tools to understand and assess engineering students’ motivation entrepreneurially minded learning.
Prof. Jacob John Swanson Minnesota State University, Mankato [biography]

Jacob Swanson is the department chair and Professor of Engineering in the Department of Integrated Engineering at Minnesota State University Mankato

Note

The full paper will be available to logged in and registered conference attendees once the conference starts on June 22, 2025, and to all visitors after the conference ends on June 25, 2025