2025 ASEE Annual Conference & Exposition

E-learning resources and educational technology are increasingly used in STEM education, generating vast amounts of student-level data. Learning analytics (LA) tools aim to utilize this data, enabling instructors to adjust their pedagogy to support both short- and long-term student success. Despite the potential benefits of data-driven instruction, the implementation of LA does not always lead to improvements in teaching practices or student performance. This paper, through two case studies, investigates how LA applications influence professional decision-making, pedagogical changes, and the challenges instructors face during this process.
In Case Study 1, we examined whether LA data accurately reflects student engagement by analyzing historical data from a learning management system alongside class artifacts provided by instructors. Descriptive codes were applied to these artifacts to assess changes in student engagement across three semesters. Online activity was compared to semester timelines and descriptive codes to identify patterns of alignment. The findings suggest that online learning analytics can serve as a proxy for student involvement, revealing engagement fluctuations across semesters. However, accurate measurement of engagement requires the integration of both LA data and contextual classroom information.
In Case Study 2, we explored how LA insights influence pedagogical change through surveys and interviews with instructors participating in an LA pilot program. Instructors generally found static data related to enrollment and academic standing more useful than dynamic data tied to students’ online behaviors. The difficulty in translating LA data into actionable pedagogical strategies rendered the current application less effective for long-term course-level improvements. While collaboration between instructors and students was recognized as beneficial, such efforts were limited. Effective adoption of LA tools requires more than just access to student data; it necessitates training and collaboration to empower instructors in making data-informed pedagogical decisions.
These case studies highlight the potential of LA tools to enhance student engagement monitoring and support pedagogical change, but also emphasize the challenges in interpreting and acting on the data. To maximize the effectiveness of LA in STEM education, institutions and departments should invest in comprehensive professional development and foster collaborative environments where instructors can share strategies and insights.