2023 ASEE Annual Conference & Exposition

Deep Learning Projects for Multidisciplinary Engineering Design Students

Deep Learning is a form of AI machine learning that has gained a great deal of recognition in the past 10 years due to numerous success stories in areas such as computer vision, language translation, face detection, medical diagnosis and treatment, visual inspection, manufacturing, etc. Deep learning is described as “end to end” learning because it encompasses low-level feature extraction and classification algorithms. This contrasts to traditional machine learning using neural networks which require a manual feature extraction step to be performed by engineers before training a neural network. Deep learning models in the area of computer vision are often implemented as a convolutional neural networks (CNN) and consist of a large number of hidden layers (10 to 100 or more). Deep learning networks, however, typically require large training databases of labeled images (10K to 1M images or more) and often require specialized GPU hardware support to train the networks. Techniques, such as “transfer learning” can often be employed to mitigate some of these requirements. Deep learning networks can often outperform human experts in many areas, such as identifying tumors in medical images or identifying defects in the manufacturing process.

This project will focus on introducing multidisciplinary engineering design undergraduates to deep learning projects in a robotics design and applications course. The course includes a module on traditional computer vision and image processing algorithms including color detection, open/close functions, noise removal, edge detection, blob analysis, Hough transforms, filtering (correlation and convolution). Deep learning is then introduced using the MATLAB Deep Learning Toolbox. with emphasis on image classification and object detection. Importantly, “transfer learning” is utilized to minimize training time and eliminate the need for GPU support. Using transfer learning, students can add customized layers to previously training networks (such as Alexnet and GoogleNet) and thereby, avoid the need for GPU support and allow the use of a smaller database of labeled images (30 to 100 images) for training.
Projects have included identification of surface defects on 3D printed objects on a conveyor belt which were then removed by a low-cost robot arm. Bottle cap defect and cap misalignment detection were also performed. A low-cost mobile robot was equipped with an ESP32-CAM wireless camera to demonstrate navigation using deep learning identification of orange cones. Other projects involving shape detection and object detection were successfully completed. The preliminary results were judged positive as indicated by student surveys.

Using the proper tools, deep learning technology is very accessible tool which can be utilized in projects by undergraduate engineering students of various backgrounds and programming skills. The paper will discuss the educational strategies used to prepare students with diverse, multidisciplinary skill sets to be able to design and deploy deep learning models. This paper will hopefully be useful to engineering educators who wish to integrate deep learning into the engineering curriculum. Due to the success of deep learning, it is anticipated that students will benefit from exposure to this exciting and highly effective technology.