2024 ASEE Annual Conference & Exposition

Abstract:
In the midst of artificial intelligence (AI) generative models becoming commonplace and widely available, academia stands at a crossroads: embrace AI or resist AI. Each institution, department, professor, and student faces this choice. AI represents an unprecedented ability to solve problems quickly with reasonable accuracy. Open AI’s ChatGPT is one such example of a generative model powered by Large Language Models (LLM). ChatGPT can solve many coding problems in a variety of computer programming languages.

Here, we seek to understand the real world consequences of embracing generative AI in general and ChatGPT in particular through the lens of coding in C++ and MATLAB. Students in the second semester of a year-long First Year Engineering class were given unlimited access to generative AI to use on homework, quizzes, and exams throughout the course. On each assessment, if they used AI, they were asked to explain how they used it.

At the end of the semester, we collected survey data covering quantitative and qualitative feedback from two sections of this class. Quantitatively, 73.2% of students said ChatGPT should be allowed in future semesters of this class. Only 5.4% of students thought ChatGPT should be banned. From thematic analysis of the qualitative survey results as well as students explanations of their use of ChatGPT on homework assignments and exams, we characterized four ways that students engaged with AI in this course:
Avoided using AI
Fixed small errors and other troubleshooting
Learned/refreshed concepts (especially when other resources were unavailable)
Solved parts of the assignment/question

We characterize some of the ways that students can use AI responsibly and as an asset to their learning without specific training. Their responses also show students’ awareness and budding understanding of the positives and negatives of AI use for acquiring and applying foundational programming skills. The results also show that the majority of students chose to use AI to enhance their learning instead of replacing their original work.