2024 ASEE Annual Conference & Exposition

Abstract
This WIP paper aims at exploring the pros and cons of using the newly released, advanced generative artificial intelligence (AI) tool, ChatGPT, to design the curriculum for a Capstone course, which is completed towards the end of the Master of Engineering Technical Management (METM), a 21-month online graduate program for working professionals in the engineering technical management fields [1].
Although generative AI technology has been around for over a decade, one could even trace relevant research back to the 1960s [2], it was the release of ChatGPT, an AI-powered language model developed by OpenAI, that brought this innovative technology into the limelight and allowed general population to access it, disrupting not only the technology sector (e.g., IT), but more recently, the academic world in terms of content generation from both the students and faculty perspectives.
This WIP paper will not dive deep into the technicality of generative AI technology- that is out of the scope of this study; but instead, it will focus on the experimental application of ChatGPT in the academic setting, to be more specific, its aid in instructional and course design.
The METM program curriculum offers courses that focus on Project Management, Strategic Planning and Management, Financial Resource Management, etc., that are included in the Engineering Management Body of Knowledge (EMBOK)[3]. At the conclusion of the METM program, students must research, design, and showcase a real-world project that requires comprehensive application of the knowledge they have learned throughout the program, in order to bring significant impact to the stakeholders of their chosen organizations.
The Capstone course spans over two semesters, Fall (Capstone I) and Spring (Capstone II); it was first offered in 2019, and in 2023, the fifth student cohort started their Capstone project. The program has been continuously evaluating and improving its courses based on stakeholder feedback, industry demands, while upholding its academic rigor. Until today, Capstone has been through three iterations of revamps: Capstone 1.0 restructured the original, Capstone 2.0 rebalanced industry-academic focus [4], and Capstone 3.0 was an instructional redesign of learning modules, which were all done through human input (subject matter expert and learners). As the course matures, fine tuning the assessments (project deliverables) has become the focus of improvement.
This WIP paper will use a case study approach to find insights in using ChatGPT to design the Capstone course. Starting from the creating the grading rubrics for one course deliverable, Project Charter, given grounding parameters to the prompt (course level, learner characteristics, generic grading categories, etc.); then, dissecting the thought process of each sub criterion to develop details of the rubric; then, asking ChatGPT to create study plan (topics, resources, activities) to achieve the high marks of the grading criteria; note that for each step, human input from Capstone faculty and instructional designer will be fed into ChatGPT to refine the prompts until a better result is generated before proceeding to the next step. Then, repeat the process for other course deliverables.
This study will keep the log of the process (prompts, refinements) mentioned above and take notes of the pros and cons of ChatGPT’s application in the course design tasks, and discuss the limitations of this approach. The result of this study will potentially lay a clearer path for other courses that would like to give this innovative technology a try for course development/improvement. For future study, we would also like to apply similar techniques to the development of new courses within the METM program.