2025 ASEE Annual Conference & Exposition

Optimizing auto-graded programming activities: A data-driven approach for presenting assessments in a scaffolded format.

Presented at Computers in Education Division (COED) Track 3.D

Research has shown that in introductory programming courses, breaking complex concepts into smaller, manageable units is highly effective. Additionally, using scaffolding techniques helps learners progressively develop programming skills. However, determining the appropriate size of each conceptual unit depends on factors such as the learners' aptitude and experience.

In this paper, we present a data-driven approach to designing auto-graded activities in our online, interactive STEM textbooks, focusing on effectively breaking down complex concepts into smaller, more achievable steps for learners. We analyzed two types of activities: 1) activities on challenging topics as reflected by high struggle rates and 2) activities on introductory topics with lower struggle rates, but where students still needed assistance based on their feedback and incorrect submissions as they began learning programming. For both types of activities we examined multiple metrics such as students' average completion rates and common errors.

Based on these insights, we further refined the activities by dividing them into smaller components and measured the impact on student struggle rates. By comparing the metrics before and after these changes, we identify key best practices for designing and improving auto-graded programming problems, aimed at enhancing student learning outcomes in programming courses.

Authors

Jamie Emily Loeber zyBooks, A Wiley Brand [biography]

Jamie Loeber is an Assessment Specialist at zyBooks, a Wiley Brand. She earned her B.S. in Computer Science at the University of California, Irvine. She has taught programing and machine learning to students across the globe. Jamie is passionate about improving computer science education and creating better learning experiences in STEM.
Ms. Efthymia Kazakou zyBooks, A Wiley Brand [biography]

Efthymia Kazakou is Sr. Assessments manager at zyBooks, a startup spun-off from UC Riverside and acquired by Wiley. zyBooks develops interactive, web-native learning materials for STEM courses. Efthymia oversees the development and maintenance of all zyBo
Dr. Yamuna Rajasekhar zyBooks, A Wiley Brand [biography]

Yamuna Rajasekhar is Director of Content, Authoring, and Research at zyBooks, a Wiley Brand. She leads content development for the Computer Science and IT disciplines at zyBooks. She leads the authoring and pedagogy team at zyBooks, developing innovative learning solutions that drive measurable student success. She is also an author and contributor to various zyBooks titles. She was formerly an assistant professor of Electrical and Computer Engineering at Miami University. She received her M.S. and Ph.D. in Electrical and Computer Engineering from UNC Charlotte.
Nicole Kehaulani Collins http://orcid.org/0000-0003-1905-284X zyBooks, A Wiley Brand [biography]

Nicole Collins is an Author Trainer and former Assessment Specialist at zyBooks, a Wiley Brand. She earned her B.S. in Computer Science and her M.Ed. in Learning, Design & Technology from UNC Charlotte. Her professional interests include computing education, online learning, educational technology, instructional design, curriculum development, and DEI in STEM. Nicole is passionate about creating engaging and effective learning experiences for students, leveraging her expertise in instructional design and technology to enhance educational outcomes for STEM disciplines.
Dr. Annie Hui zyBooks, A Wiley Brand [biography]

Annie Hui is a zyBooks assessment specialist. She has 15 years of experience teaching computer science, information technology, and data science courses, in both in-person and online modes. She has taught in Northern Virginia Community College and George Mason University. She specializes on course design to maximize student engagement and success.

Download paper (2.17 MB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session