2025 ASEE Annual Conference & Exposition

In this full empirical research paper, we aim to identify factors around students and their experiences using the [tool blinded for peer review] when answering concept questions within mechanical engineering. Instructional practices centered on active learning have been shown to positively impact student outcomes like retention, engagement, and learning gains [1] - [6]. However, Freeman et al. [1] call for “second-generation research,” where researchers should explore the relationship between instructional practices and active learning, the intensity of active learning and learning gain, or other measures related to understanding active learning and its impacts. The use of educational technology to promote active learning has been previously evaluated; however, work still needs to be done to consider instructional practices, student perceptions, and the ecosystems in which technology is being implemented [3], [7] - [10].

The [tool blinded for peer review] is a free web-based active learning tool and content repository to help instructors implement student-centered learning. The [tool blinded for peer review] currently has over 1700 faculty, 40000 students, and over 3000 concept questions in various disciplines. Concept questions, commonly called ConcepTests [11], [12], are single-right-answer multiple-choice questions with little to no math involved that ask students about fundamental concepts they are learning. The abundance of resources and community support provides instructors an accessible gateway to concept-based learning at any point in their instructional journey.

Here, we investigate the factors that impact students' experiences with active learning using the [tool blinded for peer review]. This study surveyed 448 students across a diverse set of two- and four-year institutions, asking them about their experiences using the [tool blinded for peer review] in their mechanics classes. We then use exploratory factor analysis (EFA) [13] to describe the dimensions of student experiences around the usage of the [tool blinded for peer review]. We scaffold our study with the following research question: What are the underlying factors that influence students and their experiences and perceptions around using the [tool blinded for peer review]?

We found two factors for practices, which included 1) framing and 2) effort to understand and four factors for students perceptions, which included 1) positive affect and engagement, 2) deeper learning, 3) impacts on sensemaking, and 4) negative affect. This work contributes to research that evaluates student experiences and perceptions around active learning. Furthermore, this work can help inform instructional practices for instructors who aim to integrate concept-based learning into their classes.

Keywords: Educational technology, Quantitative, Factor analysis, Conceptual learning

References
[1] S. Freeman et al., “Active learning increases student performance in science, engineering, and mathematics,” Proc. Natl. Acad. Sci. U. S. A., vol. 111, no. 23, pp. 8410–8415, Jun. 2014, doi: 10.1073/pnas.1319030111.
[2] M. Prince, “Does active learning work? A Review of the research,” J. Eng. Educ., vol. 93, no. 3, pp. 223–231, 2004, doi: 10.1002/j.2168-9830.2004.tb00809.x.
[3] Authors, “Paper blinded for peer review,” 2024.
[4] R. R. Hake, “Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses,” Am. J. Phys., vol. 66, no. 1, pp. 64–74, Jan. 1998, doi: 10.1119/1.18809.
[5] T. Vickrey, K. Rosploch, R. Rahmanian, M. Pilarz, and M. Stains, “Research-based implementation of peer instruction: A literature review,” CBE—Life Sci. Educ., vol. 14, no. 1, p. es3, Mar. 2015, doi: 10.1187/cbe.14-11-0198.
[6] D. C. Haak, J. HilleRisLambers, E. Pitre, and S. Freeman, “Increased structure and active learning reduce the achievement gap in introductory biology,” Science, vol. 332, no. 6034, pp. 1213–1216, 2011.
[7] E. C. Miller, S. Severance, and J. Krajcik, “Motivating teaching, sustaining change in practice: Design principles for teacher learning in project-based learning contexts,” J. Sci. Teach. Educ., vol. 32, no. 7, pp. 757–779, Oct. 2021, doi: 10.1080/1046560X.2020.1864099.
[8] Authors, “Paper blinded for peer review,” 2022.
[9] R. Khatri, C. Henderson, R. Cole, J. E. Froyd, D. Friedrichsen, and C. Stanford, “Characteristics of well-propagated teaching innovations in undergraduate STEM,” Int. J. STEM Educ., vol. 4, no. 1, p. 2, Feb. 2017, doi: 10.1186/s40594-017-0056-5.
[10] C. Taylor et al., “Propagating the adoption of CS educational innovations,” in Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education, in ITiCSE 2018 Companion. New York, NY, USA: Association for Computing Machinery, Jul. 2018, pp. 217–235. doi: 10.1145/3293881.3295785.
[11] E. Mazur, Peer Instruction: A User’s Manual. in Series in Educational Innovation. Prentice Hall, 1997.
[12] C. H. Crouch and E. Mazur, “Peer Instruction: Ten years of experience and results,” Am. J. Phys., vol. 69, no. 9, pp. 970–977, Sep. 2001, doi: 10.1119/1.1374249.
[13] L. R. Fabrigar and D. T. Wegener, Exploratory factor analysis. OUP USA, 2012.