2025 ASEE Annual Conference & Exposition

With the emergence of Artificial Intelligence (AI) and Large Language Models (LLMs), new approaches to conversation-based teaching have risen. Manufacturing education can benefit from AI in democratizing access to know-how rapidly, upskilling the future workforce with a high economic impact due to the multiplier effect of manufacturing. However, platforms like ChatGPT should be assessed for their capability to provide accurate and relevant manufacturing knowledge. This NSF FMRG supported paper aims to test modern AI tools by prompting three types of questions selected from manufacturing textbooks categorized as General Process, Sub-process, and Process Parameters. The prompts explicitly request answers for four different user levels: Children, Teenagers, Undergraduates, and experts. The responses generated by ChatGPT are evaluated qualitatively for correctness, relevance, and suitability for the user’s level of understanding of manufacturing, and quantitively using semantic similarity between keywords of responses and questions. Results show lower similarity for children responses indicating simpler and more abstract terms used which are more suitable for children. Yet, some responses have complex details that are not likely understandable for children. Additionally, some responses for higher levels are inconsistent and incomprehensive. Several future steps can mitigate the limitations and improve reliability of adopting current AI and LLM tools. Retrieval Augmented Generation, i.e., creating specialized Generative Pre-trained Transformer models trained on acquired manufacturing corpora results in more accurate responses with less computational cost. Integrating Visual Language Models to answer more complex queries that involve CAD models and images facilitates introducing manufacturability of a design as a fourth category of questions. Explicitly searching for analogies can lead to more effective explanation of complex responses suitable for not only children but advanced learners. User studies are needed to finetune and validate an optimized personalized conversational educational platform to train and encourage a broader population to adopt manufacturing skills.