2026 ASEE Annual Conference & Exposition

Exploring Peer-to-Peer Evaluation with an AI-Supported Learning Tool in an Introductory Programming Course: An MSI Case Study

Presented at Computers in Education (CoED): AI in Education (5 of 9) -- T408C

With the growing prevalence of generative artificial intelligence and its capacity to instantly generate computational coding solutions through large language model-based queries, concerns have emerged among educators regarding the potential attrition of students’ critical thinking and analytical problem-solving abilities. Existing education literature indicates that peer-to-peer evaluation can function as a pedagogical intervention to promote deeper learning and improve programming performance, particularly among novice programmers. These interventions provide opportunities for students to engage in delivering and receiving constructive feedback, thereby fostering communication skills, reflective thinking, and reinforcement of conceptual understanding in programming contexts. Even in the age of artificial intelligence, new tools are being developed to facilitate peer-to-peer evaluation and support collaborative learning among students.

To further investigate the impact of peer-to-peer evaluation on novice programmers, a case study was conducted in an introductory programming course at a minority-serving institution in the Mid-Atlantic United States during the Fall 2024 semester. The course integrated Kritik360, a commercial-based and AI-supported peer assessment platform, to examine the influence of structured group discussions implemented as weekly learning activities among Information Systems majors in this introductory programming course.

Kritik360 is designed to foster student engagement and promote the development of critical thinking and evaluative skills as students engage with course content through collaborative learning tasks. The primary objective of this study was to assess the effectiveness of Kritik360 in enabling students to engage in peer-to-peer evaluation within assigned groups as they develop foundational computational thinking skills and understanding core data structures in Java programming. To evaluate the impact of Kritik360 on students’ learning experiences, a survey was administered consisting of questions designed to collect both quantitative and qualitative data regarding students’ perceptions and interactions with the tool. A total of 31 students completed the survey.

The findings indicated that students held moderately positive perceptions of Kritik360’s effectiveness in enhancing their evaluative skills. Additionally, the tool was perceived as beneficial for supporting programming and code comprehension practices. However, when asked about potential areas for improvement, students noted certain aspects of Kritik360’s usability and feature capacity that could be further refined to better support the learning process.

Authors

Edward Dillon University of Maryland Baltimore County [biography]

Edward Dillon is an Associate Professor in the Department of Information System at the University of Maryland, Baltimore County (UMBC). Edward received his B.A. in Computer and Informational Science from the University of Mississippi in 2007. He would go on to obtain his Masters and Ph.D. in Computer Science from the University of Alabama in 2009 and 2012, respectively. Prior to his arrival to UMBC in August 2024, Dr. Dillon served as a Computer Science Instructor at Jackson State University (2012-2013), and a Postdoctoral Researcher at Clemson University (2013-2014) and the University of Florida (2014-2016), and Assistant/Associate Professor in the Department of Computer Science at Morgan State University (2017-2024). His research focuses on human-centered computing with emphasis on computing education, broadening participation in computing, and tech workforce prep.
Dr. Patricia Ordóñez http://orcid.org/0000-0002-5707-847X University of Maryland Baltimore County [biography]

Patricia Ordonez is an Associate Professor in Information Systems at the University of Maryland, Baltimore County since 2022. She received her BA in Hispanic and Italian Studies from Johns Hopkins University. She received her MS and PhD in Computer Science.
Carine Marette Indiana University-Bloomington [biography]

Carine Marette is a doctoral candidate in Instructional System Technology at Indiana University Bloomington's School of Education, specializing in AI-enhanced engineering pedagogy and learning analytics. Her research examines how constrained artificial intelligence environments can function as measurement affordances for observing transfer-relevant behaviors in authentic learning contexts. She developed the Vector Framework for multi-source learning telemetry, integrating behavioral data from AI interaction logs and peer assessment platforms to measure transfer readiness, productive struggle, and metacognitive development. Her work bridges engineering education, cognitive load theory, and educational measurement, with particular focus on developing validity arguments for AI-mediated assessment in project-based learning environments. Carine co-created with Kritik360 and VisibleAI platforms, which provide transparency-oriented authorship tracking and structured peer evaluation systems. Her dissertation investigates adaptive constraint design for differentiated AI scaffolding across learner distributions in graduate engineering courses.
Ben Cohen University of Maryland Baltimore County
Omobolanle Niyi Owoeye University of Maryland Baltimore County
Kevin Lemus University of Maryland Baltimore County [biography]

Kevin Lemus is a graduate assistant researcher and Master’s student at the University of Maryland, Baltimore County (UMBC). Kevin earned his Bachelor of Science in Information Systems along with a Certificate in User Experience, Web, and Mobile Development from UMBC in 2024. He is currently advancing his expertise as a candidate for a Master of Science in Human-Centered Computing, with an expected completion date of December 2026.
SRUSHTI RAJESH DHARMALE University of Maryland Baltimore County

Download paper (333 KB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session