2023 ASEE Annual Conference & Exposition

This Evaluation paper will describe the development, implementation, and evaluation for the first 2 years of a 3-year project where cohorts of high school science teachers engage in continuing education at a metropolitan-based university. The project’s primary focus is on energy, as a unifying theme across scientific and engineering domains. The project includes a robust mix of professional development, energy-related collegiate research, and energy-related industry tours; overarching goals are to: (a) deepen high school teachers’ understanding of engineering principles, practices and design, (b) support the development of STEM- and energy-integrated curriculum aligned with the Next Generation Science Standards, and (c) to enhance ongoing collaboration and interchange among university faculty, local schools, and industry-based personnel. Over the course of a 6-week summer experience, each cohort of teachers (20 teachers in the first two years, with 10 more forthcoming in year three) are paired with an engineering faculty member on one of five “energy-focused” research projects (two teachers per project – for example: Sky Radiation for Passive Space Cooling). In addition to participating in a hands-on research project, teachers also develop a curricular unit for the upcoming school year that incorporates their learning from the summer research experience with the support of study staff.

A utilization–focused evaluation approach is utilized to gather and report formative and summative data throughout the project. Both quantitative and qualitative data are collected to measure progress towards the project’s main objectives. Surveys are administered to 1) project personnel to assess the implementation fidelity of the summer workshops, and 2) teachers to measure content knowledge acquisition, satisfaction with summer workshops, and implementation of learned concepts during the school year. Qualitative analyses, based on thematic coding and cross-case comparisons, are used to analyze narrative data obtained from recurring, yearly focus groups and open-ended items included in surveys. More specifically, focus group interviews are utilized to diversify and expand upon the study’s emergent themes as teachers transition from summer research experience to curriculum implementation in the classroom.

Two commonly used instruments are utilized prior and after teacher immersion in the summer research experience to measure their orientations and beliefs: 1) a (PRE/POST) card-sort task to elicit and clarify engineering/science-teaching orientations and 2) modified teacher beliefs about effective science/engineering teaching instrument. “Quick Cards” and end-of-day evaluations are used to evaluate the effectiveness of teacher summer experiences. These evaluative measures provide participants means in indicating the effectiveness in 5 domains: (a) content focus (relevance for practice); (b) opportunities for active learning; (c) coherence with the school/district curricula; (d) duration (adequacy of time for a given segment); and (e) collective participation peer support. Each measure is used iteratively to improve each subsequent year of the project. Finally, two (end of fall/spring) iterative assessments of teacher developed curriculum units occur using the STEM Integration Curriculum Assessment (STEM-ICA). This curriculum assessment is used in a formative and evaluative manner.