2024 ASEE Annual Conference & Exposition

Building on work presented at ASEE 2023, this NSF Grantees Poster Session paper reports on the impacts of the [NSF Project]—a $1M NSF award via the NSF ECR-EHR Core Research program in 2019—as it nears the end of its final year. The [NSF Project] aims to build national capacity for STEM education research by engaging technical STEM from across the U.S. in cohorts that participate in a semester-long course on qualitative and mixed methods educational research techniques. Faculty from underrepresented backgrounds and Minority-Serving Institutions were given priority consideration in terms of recruitment and admission to participate in the project. This project was funded based on impact rather than knowledge generation; thus, this paper will report on the [NSF Projects]’s summative outcomes and impact revealed through external evaluation.

Using the Qualifying Qualitative Research Quality (Q3) framework pioneered by Dr. Joachim Walther and colleagues as a foundation, the project team guided three cohorts of faculty (48 faculty total) in designing qualitative or mixed methods studies to address research questions they wanted to answer about their educational contexts. The project team has also hosted four follow-up research incubators (each one semester long) as spaces to allow graduates of the [program] to continue working together as a community to continue developing educational research projects to completion or apply for extramural funding opportunities (serving 27 faculty total.) Furthermore, the team has funded graduates of the [program] to lead communities of practice focused on areas of shared research interest among graduates. Finally, in the past year, the team has funded professional grant proposal coaching for interested graduates and has worked with graduates to design and execute workshops on the [program’s] approach to qualitative research design.

The evaluation results presented in this paper derive from a primarily quantitative, mixed methods survey conducted by the project’s external evaluators. The survey aimed to gauge participants' engagement with [program] activities; their knowledge, perceptions of, and comfort with qualitative and mixed methods; assessment of training outcomes and curriculum; the influence of [program] training on research dissemination; and self-reported behavioral changes before and after training. Results indicate substantial self-reported improvement in knowledge of qualitative research methods, as well as application of those methods and confidence to do so. Many participants have gone on to pursue funded research projects using qualitative and mixed methods research approaches, and many have reported increased capacity to collaborate on and disseminate research projects successfully. Constructively, the results point to a need to support nascent qualitative researchers in attending educational research conferences and integrating into these conferences’ communities. Participants also indicated a desire for more practical experience with real-world qualitative research scenarios, pointing to a need for further hands-on professional development opportunities. Overall, the survey paints a picture of a successful program that nearly all respondents reported as being worthwhile and a desire from participants to continue seeking opportunities to grow as qualitative and mixed methods researchers.