2023 ASEE Annual Conference & Exposition

Buffer overflow is one of the most common vulnerabilities reported by the Common Vulnerabilities and Exposures (CVE) program. Giving students a mental model of how buffer overflow works and how dangerous these vulnerabilities are may instill in them a commitment to avoiding these vulnerabilities in the future. Buffer overflow and stack frames are known to be a difficult concept for students to understand. This experience report describes a buffer overflow assignment in which students use a free and open-source emulator and debugger to exploit a buffer overflow and view the effect on stack memory. The assignment steps students through assembling a C program vulnerable to buffer overflow in the emulator, running the program in the debugger, exploiting the vulnerable function and causing a buffer overflow, then examining the registers and the stack memory to see the effect of the buffer overflow. Students are then guided through overwriting a function return address saved in the stack with a buffer overflow and causing arbitrary code to run. We provide the assignment instructions and questions that students must answer. 591 students in a Computer Organization course in Fall 2022 were required to complete the assignment. Over 90% of students could identify input data when viewing stack memory. 80% could identify that buffer overflow caused a saved register to be corrupted and cause the program to crash. 64% of students could identify saved registers in stack memory and 42% of students could correctly interpret the boundaries of stack frames. These results give insight into the effectiveness of the assignment and which parts are most difficult for students to understand. Students also responded to the reflection prompt “What was the most surprising or interesting part of this activity.” The responses were analyzed for common themes, which were the usefulness of visualizing memory in understanding the concepts of stack frames and buffer overflow, the prevalence of buffer overflow vulnerabilities in publicly available code, and how easy it is to exploit a buffer overflow vulnerability. Thus, this assignment shows promise in helping students to understand a difficult concept, and in emphasizing the importance of avoiding buffer overflow vulnerabilities.