2024 ASEE Annual Conference & Exposition

The overall goal of this project was to establish an NSF REU site that integrates undergraduate students in team-based research projects focused on converting biological wastes into products of value. This site has operated for two summers at Auburn University and has graduated 19 REU fellows from diverse backgrounds. Each fellow was paired with a faculty mentor but conducted research on an interdisciplinary team-based project. We hypothesized that this team-based approach would improve confidence and knowledge regarding effective team and research practices. The objective of our project evaluation was to determine how the REU site affected fellow confidence in effective team and research practices. A summative assessment of the project consisted of a pre-post survey instrument focused on perceived self-efficacy in universal teamwork and research skills. This instrument was an adaptation of the Research Self-efficacy scale. Of the 19 REU participants to date, 17 signed the letter of consent to participate in the IRB-approved evaluation research. However, only 10 chose to complete both the pre- and post-assessment instruments. In the first year, we were surprised to see that fellows actually scored lower or showed no change in multiple aspects of research self-efficacy after participation in the program. This data contrasted with weekly qualitative survey data that suggested that fellows had learned a great deal about research from the program. Upon review of the pre- and post-test data, we determined that the disconnect between quantitative and qualitative data may have been related to vague and sometimes programmatically misaligned measure items in the Research Self-efficacy scale (which uses a Likert scale). From focus groups conducted with fellows at the end of the experience, it was apparent that fellows had learned enough about research that they could identify how much more they had to learn to become experts. To better quantitatively measure fellow perceptions of learning, we modified the pre- and post-test survey statements to assess research fellows' self-efficacy in various research-related skills, such as understanding research components, data analysis, and independent project work.. Across all summative questions, we saw an average score increase of 12% in year 1. The percent improvement was determined as the change in score for the question divided by the highest point value on the Likert scale. In year 2, with the revised set of questions, the average score increase across all questions was 19%. After revising to address year 2’s data, the pre- and post-test will be used again in year 3 to see if similar levels of improvement are achieved again. In year 3, positive incentives will also be used in an attempt to increase participation and completion of the survey instrument, guided by a commitment to ethical standards, transparency, fairness, and protecting participants' rights and privacy. We note that the limited sample size, resulting from incomplete survey participation and a small study population, may affect the findings' generalizability. This research informs and enhances assessment of undergraduate research experiences and provides a framework for continuous program improvement.