2025 ASEE Annual Conference & Exposition

Integrating Image, Video, and Machine Learning into an IoT Learning Environment

Internet of Things (IoT) technology continues to enable the design and development of a variety of new types of applications and systems. The advanced capabilities found in a modern smart home provide numerous examples including the remote and automated management and control of thermostats, lighting systems, and security devices. More broadly, the data gathering and control capabilities of IoT technology is also helping to fuel the development of significant improvements of existing larger scale infrastructure related systems for important tasks such as traffic management (smart signals) and power distribution (smart grids). As a result, the importance of teaching IoT related concepts and technology to students in computer science, electrical engineering, computer engineering and other relevant STEM education programs continues to increase. As graduates from these programs enter the workforce they will require knowledge of sensing devices, communication technologies, and control techniques to successfully meet an ever-increasing demand for the design and support of IoT related systems.

An ongoing project at Texas A&M University-Kingsville and Texas A&M University-Corpus Christi, both Hispanic Serving Institutions, has focused on enhancing support for teaching IoT related concepts and technology. The project has especially emphasized expanding support for engaged remote student learning to accommodate learners that do not have ready access to lab facilities, such as those with family or work obligations that make it difficult to attend class at a time when labs are typically scheduled. A learning toolkit approach has been adopted in which a collection of basic IoT equipment and components has been assembled that can be utilized by remotely learning students to complete IoT related exercises and assignments. Students are provided with a learning toolkit at the start of a class and can then utilize it throughout the semester to study IoT related materials at a time and place that fits their schedule.

The initial IoT learning toolkit designed during the project was based on a simple processor board. A set of exercises was developed to guide students through the process of connecting the necessary components of the toolkit to establish and test a basic IoT learning environment. Subsequently a second more extensive toolkit was designed based on an IoT learning platform. A second series of exercises was also developed to introduce students to the components and capabilities of the learning platform and how they can be utilized in designing IoT solutions. There are advantages and disadvantages associated with both versions of the IoT toolkit, and a student’s technical background as well as the complexity of an IoT exercise or assignment to be performed are significant factors in choosing between them.

More recently a new version of the IoT learning toolkit has been designed to incorporate a camera module. A set of exercises is being developed to accompany the new toolkit and teach students how images and video can be utilized in IoT solutions. The exercises will guide students in learning how to integrate the camera module into an IoT solution and capture images and video for local storage or to be sent to a cloud-based facility for storage and analysis. Students will also learn about strategies for exploiting image recognition and other machine learning techniques in the design of an IoT solution. The current paper provides a detailed look at the components included in the augmented tool kit. It also examines the tasks associated with the new exercises and how they will be used to teach students about the opportunities for utilizing images, video, and machine learning in an IoT solution.