2026 ASEE Annual Conference & Exposition

Benefits of Critiquing a ChatGPT Generated Essay: Preliminary Results and Insights

Presented at Industrial Engineering Division (IED) Technical Session 2: AI Integration and Academic Integrity

ChatGPT has been increasingly popular with students since it became publicly available in 2022. There are lots of different ways ChatGPT and other similar tools can be utilized to improve higher education, but it is important to make sure students use it carefully. To provide a structured introduction to the capabilities of ChatGPT, we designed an assignment for an undergraduate level supply chain course, asking students to work with ChatGPT to create essays and then to analyze them. Each student prompted ChatGPT for the essays using prompts they created based on the assignment requirements, asking them to analyze 160- to 180-word essays that included citations and references. Students then had to complete a sentence-by-sentence analysis to look for both conceptual and grammatical errors. At the end of the semester, students were then asked to reflect on this experience, to gain an understanding of its impact on students' comprehension of the topics as well as understanding of the capabilities of ChatGPT to aid in their education. This study will look at data from the Spring 2024 semester to investigate how students analyzed ChatGPT’s work along with their observations about their experiences with the assignment. There were 38 undergraduate industrial and systems engineering students in the course with 31 consenting to having their submitted assignments and end-of-course survey responses used in research. Ultimately, this paper will outline the general impact this assignment has on students and an analysis of essays written by ChatGPT and students’ analyses.

Authors

Zoe Long University of Florida [biography]

Zoe Long is a PhD student at the University of Florida. They are interested in research about how to improve engineering education in creative ways. Currently, they are working on how large language models can be utilized to improve engineering education with a focus on the student perspective.
Golbarg Nazari University of Florida [biography]

Golbarg Nazari is a Ph.D. student in Industrial and Systems Engineering at the University of Florida. Her research focuses on AI in education and engineering education, with particular interests in student motivation, creativity, and human-centered approaches to learning. Her work examines how innovative instructional strategies, including GenAI-supported, reflective, and creative activities, can enhance student engagement, deepen conceptual understanding, and promote meaningful learning in engineering education.
Addison Grayce Lee University of Florida [biography]

Addison Lee is an Industrial and Systems Engineering undergraduate student at the University of Florida. Alongside her studies she assists Zoe Long and Dr. Akçali in their research efforts through data organization and analysis. Her interest in the complex nature of artificial intelligence, specifically how its usage can both aid and hinder society, has led to her current research in the usage of large language models in the classroom.
Prof. Elif Akçalı University of Florida [biography]

Dr. Elif Akçalı is an Associate Professor in the Department of Industrial and Systems Engineering as well as The Michael Durham Professor in Creativity in the Engineering Innovation Institute in the Herbert Wertheim College of Engineering at the University of Florida. She is an industrial engineer, a visual artist, and an explorer of the interplay between thinking and making in the arts and engineering. Her scholarly interests focus on the use of arts-based processes for the development of divergent thinking and critical thinking skills of engineering students.

Download paper (460 KB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session

For those interested in:

undergraduate