2025 ASEE Annual Conference & Exposition

Exploring the Effective Use of ChatGPT in a Sophomore-Level Dynamics Course

The rapid progress in the field of Artificial Intelligence and the swift adoption of these tools by students merits a close look at the use of chatbot-style tools to solve engineering problems of all types. This study investigated the performance of three existing chatbot tools, ChatGPT, Claude, and Gemini, in solving Dynamics problems. Specifically, the Dynamics Concept Inventory was used as an assessment of the chatbots’ ability to answer conceptual problems in a multiple-choice format when only provided with images of the problem (no text prompts). It was found that while the current version of chatbots performed well in the overall task of understanding the question and providing a high-level problem-solving strategy, they suffered from errors in visualizing spatial relationships and in reasoning about primary principles.

Authors

Ryan Carr U.S. Air Force Academy [biography]

Ryan Carr received his PhD from the Air Force Institute of Technology (AFIT) in 2017 focused on optimal control theory in guidance, control, and navigation or aerospace systems. He was an engineer and branch chief at the Air Force Research Laboratory (AFRL), a flight test engineer the Air Force Test Center (AFTC), and a program manager at the Air Force Office of Scientific Research (AFOSR). He joined the faculty at the United States Air Force Academy in 2023 as an Assistant Professor of Mechanical Engineering.
Dr. Phillip Cornwell U.S. Air Force Academy [biography]

Phillip Cornwell is a Professor of Mechanical Engineering at the United States Air Force Academy and an Emeritus Professor of Mechanical Engineering at Rose-Hulman Institute of Technology. He received his Ph.D. from Princeton University in 1989, and his current interests include structural dynamics, structural health monitoring, and undergraduate engineering education. Dr. Cornwell received the SAE Ralph R. Teetor Educational Award in 1992, the Dean’s Outstanding Teacher Award at Rose-Hulman in 2000, and the Rose-Hulman Board of Trustees' Outstanding Scholar Award in 2001. He was one of the developers of the Rose-Hulman Sophomore Engineering Curriculum and the Dynamics Concept Inventory. Additionally, he is a co-author of Vector Mechanics for Engineers: Dynamics, by Beer, Johnston, Cornwell, and Self. In 2019, Dr. Cornwell received the Archie Higdon Distinguished Educator Award from the Mechanics Division of ASEE, and in 2023, he was honored with the Ralph Coats Roe Award from the Mechanical Engineering Division of ASEE.

Note

The full paper will be available to logged in and registered conference attendees once the conference starts on June 22, 2025, and to all visitors after the conference ends on June 25, 2025