2025 ASEE Annual Conference & Exposition

Broadly, metacognition means “thinking about thinking.” Although the specific elements of metacognition vary across the literature, some commonly accepted metacognitive activities include planning, self-monitoring, and self-reflecting (e.g., Zimmerman’s Cyclical Phases Model). Previous research has identified a relationship between practicing metacognition and improved student outcomes, including higher achievement, better learning strategies, and improved programming skills.

However, metacognitive strategies are rarely taught explicitly in higher education and despite foundational research on metacognitive skills, empirical interventions of metacognition are lacking. Most tools primarily focus on offering cognitive activities, such as review activities or coding exercises. Furthermore, existing metacognition-based interventions do not capture the entire cycle of studying (planning, self-monitoring, and self-reflecting).

Therefore, we developed and piloted [Anon Tool], a web-based application that integrates four metacognition-based features used in prior research. They are - the planning prompt, quizzing, reflection prompt, and evaluation prompt features. The first and the last feature promote the planning and self-reflecting phase respectively while the middle two promote the self-monitoring phase. This way, [Anon Tool] targeted all three phases of metacognition. The guiding questions of our intervention can be formulated as follows: 1) Do students actually engage with different metacognition-based features throughout a course?; and 2) How is student engagement with different metacognition-based features related to their learning outcomes?

We deployed [Anon Tool] to students in a Data Structures and Algorithms (CS2) course at a public R1 university. To ensure that students understood how to use [Anon Tool], we held a mandatory, one-time “workshop” where we demonstrated how to use our tool, explained the metacognition cycle model, and briefly presented some research on the empirical benefits of practicing metacognition. Subsequently, we collected data on student usage of the various features of [Anon Tool] throughout the term.

Our database records show that students engaged the most with the quizzing feature. Usage of the features including planning prompt, reflection prompt, and evaluation prompt was much lower. Additionally, feature usage spiked significantly during the two midterm weeks. Still, our findings indicate that feature usage on [Anon Tool] and learning outcomes are correlated. Long-term users of the quizzing feature outperformed short-term users on midterm 2 and demonstrated higher and more concentrated first-try correctness. For both the planning prompt and reflection prompt feature, long-term users performed better than their short-term counterparts. No significant analysis could be performed for the evaluation feature due to the small sample size.

Such correlation between the feature usage of [Anon Tool] and learning outcomes implies the potential of [Anon Tool] as a measurement tool that reflects students’ level of metacognition. Thus, our future work could be 1) how to encourage students to use [Anon Tool] more frequently and 2) how to further measure such behavioral data more accurately. [Anon Tool] is publicly available at [REDACTED] and we have designed the tool such that new instructors can manage their own classes and set of questions. We encourage instructors interested in using the tool to reach out to the authors of this paper.