2024 ASEE Annual Conference & Exposition

Funded through the NSF ITEST program, the primary objective of this mixed-methods meta-analysis and qualitative synthesis study is to review and synthesize research and evaluation findings demonstrating the effects of integrating innovative technologies and technology-based learning experiences in STEM education on K-12 students’ STEM career-related outcomes. This study synthesizes the rigorous intervention research on Grades K-12 students’ STEM career-related outcomes from 1995 to the present and across characteristics of innovative technology-based STEM education interventions, learning contexts, student demographics, and study designs. This study advances understanding of effects of integrating innovative educational technologies and technology-based learning experiences in PreK-12 classrooms on students' STEM career outcomes, enables generalization of the magnitudes and variations of effects on students, what settings specifies what technologies and interventions have been effective for which groups of students and in, and provides insight to how and why such interventions produced positive outcomes. The method for conducting this study follows steps common to meta-analysis and qualitative synthesis studies, that is, establishing inclusion/exclusion criteria and search terms following the PICOS framework (i.e., participant, intervention, comparison condition/study design, outcome, and setting), conducting database searches, screening for study inclusion, coding, and conducting analyses.

The preliminary meta-analysis involved 94 effect sizes from 42 primary studies, most published within the past decade. These studies encompassed 13,069 student participants, primarily in middle or high school. STEM career-related outcomes were measured as dispositions, including interest, aspiration, motivation, confidence, and self-efficacy. A small number of studies also assessed knowledge in specific STEM careers. Overall, a small positive effect was observed (effect size mean = 0.189, SE=0.046, 95%CI = 0.099 – 0.280, p < .001), with significant heterogeneity (Q = 932.51, p < .001, I2 = 0.94), suggesting the need to explore potential moderator variables. Intervention characteristics revealed that 48% targeted underrepresented and/or underserved populations, 40% included explicit career development, and interdisciplinary content was common. Additionally, 60% of studies took place in informal settings. The study also considered intervention format, duration, pedagogical practices, study design, and publication type as potential moderators in the final analysis.

The preliminary qualitative synthesis revealed that 21 of 27 studies reported student outcomes along at least one of two trajectories: prior STEM career interest sustained or strengthened after participation in the intervention (“STEM-supported”) or STEM career interest piqued by participation in the intervention (“STEM-Influenced”). Because these trajectories were not clearly associated with potential moderators, more analysis is required to further explore the extent to which combinations of moderators are more or less associated with different STEM career development trajectories, including trajectories that did not result in greater interest in STEM career fields.