2023 ASEE Annual Conference & Exposition

Modern educational assessment methods for computing that measure computer science (CS) students’ learning of programming have increasingly moved to online or computer-based testing formats with recent advancements in educational technologies. Such methods to test a students’ aptitude include browser-based applications where students receive immediate feedback on code, cloud-based autograders, online exams, and applications installed on remote virtual machines that provide students’ feedback via testing libraries. With rising enrollment in computing, we need to ensure that assessment methods accommodate learners with different needs and preferences. This paper aims to describe the needs, preferences, and the barriers of students as they write short programs that provide them with feedback. Our data is collected through a survey which follows students’ interaction with our web-based drill and practice programming system called Anonymous in the context of a Data Structures and Algorithms (DSA) course at a large public university in the United States. Our system provided students two workflows for solving and receiving feedback on short programming problems: (1) using a browser-based workflow and/or (2) downloading an equivalent template of the problem and feedback, and coding it locally on their computer (a native workflow). We qualitatively coded 199 students’ responses regarding choices using inductive thematic analysis to identify common themes. Our study found that while most students were motivated by convenience and ease of use to solve programming problems in browser-based systems, there is a specific cohort that prefers to program locally on full-fledged Integrated development environments (IDE) due to limitations in systems designed for hosting short programming problems such as lack of debuggers, familiarity with used IDEs, etc. Our research has implications for computing educators, system designers, and other stakeholders involved with the design or selection of programming environments or workflows who want to accommodate eclectic learner needs and support students to code natively or in browser-based solutions.