2025 ASEE Annual Conference & Exposition

This paper explores the effect of autograders on student coding abilities, coding confidence, and overall learning experience in a junior-level mechanical engineering class on numerical methods with individual assignments and an open-ended collaborative term project. In coding-intensive large enrollment courses where human grading might take several weeks, autograders can provide immediate and consistent feedback. The real-time feedback allows students to learn from their mistakes and rectify those mistakes, thereby improving understanding and coding confidence. Autograded assignments can also reduce the grading time and effort for graders and teaching assistants in addition to lowering the burden for instructors to provide quick feedback on intermediate attempts of student codes during office hours. However, existing studies highlight potential concerns such as fostering students’ overreliance on the autograder instead of encouraging independent debugging efforts. Additionally instructors using autograders must provide additional instruction with the assignment so that students are able to effectively cater to the autograder requirements. In this study, we aim to analyze whether the availability of an autograder implemented with Gradescope and developed for mechanical engineering students with limited formal coding exposure can improve student coding confidence, perceived coding ability, and student engagement. Additionally, we investigate whether the autograder reduces debugging time on assignments, promotes over-reliance on the instant feedback, and whether hidden tests for some tasks could mitigate any overdependence. Preliminary results from an end-of-semester survey administered to students suggest that the autograder increased student coding confidence and abilities, helped them remain interested in project goals and deliverables, and reduced debugging time before submission. Additionally, students reported that the autograder on individual assignments helped them learn the basics of Python programming and fundamentals of mathematical concepts taught in the course. Interestingly, students reported that the hidden autograder tests did not help them attain independent debugging skills, but student confidence was largely unaffected by removing the autograder in subsequent assignments.