2024 ASEE Annual Conference & Exposition

In this full paper, we describe our case study approach to initially characterize Design Cohesion as a new IDE-based learning metrics through an exploratory coding process. We developed a new alignment notation to generate two new qualitative metrics: Design Cohesion (High, Medium, Low) and Granularity Level (High, Medium, Low) Design Cohesion is the level of alignment between flowchart and code, which accounts for the order of intended program execution, the internal data of a flowchart node, and the number of nodes in the flowchart that map to the code. We define Granularity Level as an additional characterization of cohesion that labels the level of detail in the flowchart. Our primary objective is to use these new metrics to understand how a flowchart can be aligned with its code implementation to understand introductory students’ current level of programming metacognition.

In the context of our case study, we discuss the exploratory coding process and alignment notation developed to generate the features for the newly proposed metrics. Next, we explore two cases to illustrate the diversity of characteristics found in various feature combinations. Each case study compares two examples from the same participant, one with High Cohesion and High Granularity, the other with High Cohesion and Low Granularity.
Our initial investigation into design cohesion has led to the hypothesis that High-level Design Cohesion paired with low levels of flowchart granularity demonstrates high levels of abstraction in the initial flowchart design, which may point to under-designing by participants and/or lower levels of metacognition. Comparatively, having high cohesion and granularity may point to over-designing by the participant and often stems from a one-to-one mapping of flowchart nodes to lines of code. Our results point toward a logical relationship between Design Cohesion and students’ level of self-estimated skill, and we are confident that Design Cohesion will serve as viable metric for understanding introductory programming metacognition.