2023 ASEE Annual Conference & Exposition

A rubric is presented to assess debugging skills for students particularly in the natural sciences and engineering. The three categories that are assessed for the cognitive processes in debugging skills are identification, isolation, and iteration. These are defined, and the characteristics of each process are listed. We discuss the method used to develop this rubric that was based on intentional errors in a programming assignment given to students in an introductory physics course. The programming in this assignment was in Python and a visual-based programming platform, called iFlow. We believe that visual-based programming will help elicit weaknesses in debugging because it removes students' familiarity with particular programming languages.

Our focus on debugging skills came from a survey of students to self-identify barriers in computational work in an introductory physics course that included engineering majors. This skill was the primary self-identified barrier along with abstraction skills, which will be the focus of another work. We also present the results of this survey. The Python assignment (ntext = 9) was used to create the rubric and the iFlow assignment (ngraphic = 11) was used to test the rubric. Scoring was based on a scale of six levels in each category. Although the sample size was too small to establish rigorous scoring reliability, we discussed how the two researchers attained agreement in scoring the assignments after iterative modifications of the rubric and rescoring.
For the Python assignment, the average for identification was 2.75/5, for isolation 2.30/5, and for iteration 3.33/5. For the iFlow assignment, the average for identification was 2.63/5, for isolation 2.23/5, and for iterate 3.32/5. A consistent trend from these assignments showed that students' approach to debugging is mainly to identify and iterate without a full understanding of the error (i.e., isolation). The lack of a full understanding of the error implies that students are prone to repeat the error. Thus, the important outcome of debugging is to understand the source of error by systematically investigating different parts of the computational solution. Our preliminary results led to the hypothesis that students with weak debugging skills are mainly due the isolation process. This hypothesis will be tested in a future experiment. Results from such an experiment will be significant to those who are designing intervention strategies to integrate computational thinking in science and engineering curricula.