2026 ASEE Annual Conference & Exposition

WIP: Self-reported certainty in engineering students' knowledge of bioinstrumentation and validity of statements from peers and LLMs

Laboratory courses seek to translate classroom knowledge to applications in the real world. In that translation, students must grapple with complex problems where they learn that there is more to learning than simple factual recall. The ability to gauge their own confidence in knowledge, or the ability to admit when they don't know something, is essential to their future career, when their work will be relied upon to improve or save lives. When students are asked to think about how they know something and whether they are certain about it, they explore their thought processes critically and deepen their learning. We have previously done work on students’ confidence in their ability to learn relevant material for a bioinstrumentation laboratory course, and we have improved the feedback methods to better provide scaffolded, formative assessment. This work builds on both of these concepts to develop a model of student certainty, or their self-evaluation of performance on a specific knowledge task, while also studying student epistemologies in engineering. We are implementing this in a laboratory course setting because there is often a gap between pre-lab assignment knowledge and the critical thinking necessary to use that knowledge in an instrumentation context.
The purpose of this study is to develop a formative assessment activity that introduces students to the concepts of certainty and accountability in engineering practice, while also confronting them with statements of varying subjectivity to prepare them for poorly-defined problems. Students are given short ungraded quizzes at the beginning of their lab section, which ask them to respond with an answer to a question, an assessment of their certainty in that answer, and an evaluation of the validity of a statement derived either from a peer or an online source. This is aligned with the course objective of promoting critical thinking about course content, but will also provide quantitative data on how reported certainty is affected by the source of the statements they evaluate. By asking the same question in three different ways, we can determine if students react differently to information that comes from their own memory, a peer, or a Large Language Model. We will determine whether student certainty correlates with characteristics like declared major or reported gender, and whether certainty correlates with correctness. A final survey will also ask students about their relationship with epistemology and how their critical thinking skills are affected by their engineering coursework.

Authors
  1. Joseph Tibbs Orcid 16x16http://orcid.org/0000-0002-6315-9778 University of Illinois at Urbana - Champaign [biography]
Note

The full paper will be available to logged in and registered conference attendees once the conference starts on June 21, 2026, and to all visitors after the conference ends on June 24, 2026