Login

2026 ASEE Annual Conference & Exposition

Impact of the FossilSketch Application on Students' Use of Generative Artificial Intelligence for Conceptual Assessment

Presented at Computers in Education (CoED): AI in Education (6 of 9) -- T508A

Prior research suggests an increasing reliance of undergraduate students on Generative Artificial Intelligence (GenAI) tools. As part of a larger project to develop and evaluate the effectiveness of the FossilSketch, a web-based learning tool, this paper examines how GenAI use changed before and after using the app. For this study, we hypothesized that effective learning environments, such as the FossilSketch, can impact (i.e., lower) the use of GenAI. For this purpose, we introduced the FossilSketch, designed to teach micropaleontology concepts to diverse students in three courses. Although all three courses were undergraduate general science courses, they varied in teaching modalities (in-person or online), nature of lab sessions (no labs, or required labs), level of coverage of micropaleontology concepts before FossilSketch (none, partial through guest lectures, or frequently through lectures), and the way the FossilSketch was introduced (extra-credit assignment or required assignment). Considering these variations, each course offers a unique perspective on the use of FossilSketch.

In all three courses, students completed a pre- and post-conceptual assessment comprising 4 open-ended constructed-response questions. For the pre-assessment, the researchers identified commonalities between student responses and GenAI responses. The researchers also identified writing characteristics that helped classify students' responses as either GenAI-generated or student-generated. To explore the change from before and after the use of the FossilSketch, this paper addresses two research questions: 1) How does the use of GenAI vary across courses before and after the use of the FossilSketch application? 2) How does the use of GenAI vary between questions before and after the use of the FossilSketch application?

In this cross-sectional study, we collected data from 111 undergraduate students (n1=40, n2=36, n3=35) at a large R1 university. To prepare the data, we classified each student's response as either student-generated (0) or GenAI-generated (1) for both pre-assessment and post-assessment. The data were analyzed using descriptive statistics and chi-square tests of students' use of GenAI in pre- and post-assessments across three courses. Also, chi-square tests were used to examine students' use of GenAI across questions in the pre- and post-assessments. Results indicated that the number of students who used GenAI was significantly lower in the post-assessment than in the pre-assessment. Also, we found that the variation at the question level was similar across courses. In line with our hypothesis, this study's results indicate that authentic context and activities can influence students' decisions to exercise their own judgment and become self-reliant. The study explains the need for additional research to examine how integrating active learning across different types of courses can mitigate factors that could push students to become overly reliant on GenAI tools.

Authors
  2. Dr. Daniel Sungje Bahng Texas A&M University [biography]
  4. Christina Belanger Texas A&M University
  6. Katherine Sarah Cherry Texas A&M University
  7. Syeda Fizza Ali Texas A&M University [biography]
Note

The full paper will be available to logged in and registered conference attendees once the conference starts on June 21, 2026, and to all visitors after the conference ends on June 24, 2026

« View session

For those interested in:

  • computer science
  • engineering
  • information technology
  • undergraduate