2026 ASEE Annual Conference & Exposition

This paper presents a sequence of themed labs and a culminating competition for an advanced
multidisciplinary engineering course that bridges theoretical foundations in signal processing and
machine learning with hands-on design of embedded, intelligent systems. Using the ESP32
microcontroller, students progressively engage in projects that introduce embedded machine
learning (ML), human–computer interaction (HCI), hardware–software co-design, and
edge–cloud integration. The course was designed to make those concepts accessible to graduate
students from a range of interdisciplinary majors.
The first lab focuses on building a “magical sorting hat”, inspired by the Harry Potter universe,
that classifies students into different houses based on their characteristics. Through this exercise,
students gained hands-on experience with buttons as input devices, data collection, and
preprocessing techniques essential for supervised learning. They implemented a decision tree
classifier and deployed it directly on the ESP32, while also creating an interactive interface with
an OLED display. This project highlights the process of designing a playful yet structured
classification system that integrates hardware control, software logic, and machine learning on a
resource-constrained platform. By completing this lab, students strengthen their skills in
prototyping embedded systems and in creating user experiences that blend technical rigor with
creativity.
The second lab expands on these skills by implementing a gesture recognition system using an
ESP32 microcontroller paired with an MPU6050 inertial measurement unit (IMU). In this project,
students learned to capture time-series gesture data and preprocess the data effectively to ensure
reliable classification. Using the Edge Impulse platform, they apply digital signal processing
(DSP) techniques and lightweight machine learning models optimized for embedded devices.
This lab emphasized the integration of hardware and software in a feedback loop, where proper
sensor calibration, data quality, and inference accuracy all directly affect the user
experience.
The last lab extends the wand into an edge–cloud hybrid system, where inference occurs locally
on the ESP32 when confidence is high, and offloads to Microsoft Azure when uncertain. Students
train and host models in Azure, deploy web apps as inference endpoints, and implement fallback
logic for cloud consultation. This lab allows students to grapple with issues of connectivity,
latency, consistency, and data privacy, developing interdisciplinary solutions and design strategies
that highlight the complex tradeoffs inherent in modern intelligent systems.
Finally, a wand duel competition integrates all prior work. Each wand reports recognized gestures
and confidence scores to a central server, which applies predefined “spell rules” and
house-specific abilities to determine duel outcomes. The competition reinforces system
integration, communication protocols, and design tradeoffs.
Together, these labs provide a comprehensive introduction to embedded machine learning and
interactive systems. Students develop skills in cross-disciplinary communication, systems
integration, and design reasoning, learning to translate between algorithmic, hardware, and
user-experience perspectives. By combining creativity with technical depth, the labs prepare
students to design intelligent, user-centered systems that demonstrate both the potential and
challenges of deploying machine learning at the edge.