2023 ASEE Annual Conference & Exposition

This theory paper describes challenges and opportunities with analyzing engineering curricula using the Curricular Analytics framework by offering a data collection framework for systematically collecting plans of study at scale. Introduced by Heileman and colleagues in 2017, the Curricular Analytics framework enables researchers and practitioners to quantify the interconnectedness of their prerequisite structures to unveil gatekeeper courses and forecast the impact of curricular policies or changes using network-analytic metrics. These metrics can be calculated using available data; all one needs to do is transform a plan of study into a list of courses, prerequisites, and corequisites.

Although the data requirements are minimal, there are distinct challenges to employing this framework across disciplines and institutions at scale – especially if longitudinal analyses are planned. For example, curricula are not always completely defined, leaving space for students to select electives with varying degrees of flexibility. Without specifying these electives, the curriculum’s complexity may be underestimated. Moreover, there are deeper data entry considerations; prerequisites can often be a complicated series of ANDs and ORs, with language like “at least” or “X of the following.” These configurations do not lead to obvious network representations. Finally, finding accurate plan of study information, even for recent years, can be challenging. Prerequisite and course information can be inaccurate or may not have been appropriately maintained by the respective institutional office.
This paper focuses on standardized procedures and conventions employed in response to obstacles encountered during the first year of a project to collect plans of study for five engineering disciplines across the US: Civil Engineering, Electrical Engineering, Mechanical Engineering, Chemical Engineering, and Industrial Engineering. The sampling frame, in this case, was the Multi-Institution Database for Engineering Longitudinal Development (MIDFIELD). We chose to sample the most recent ten years since the last record per institution. To synthesize a standard operating procedure for collecting plan of study data at scale, we reflected on the processes taken during our initial data collection. The data sources we drew upon were (1) written records from weekly meetings; (2) the Microsoft Teams chat log for the project where personnel collaborated and posted questions for one another; and (3) reflections from the five undergraduate research assistants. We applied the constant comparative technique to find common data collection and entry issues, resulting in a flow diagram for standardized data entry usable across majors.

We anticipate this work being useful to researchers and practitioners interested in systematic analyses of curricula, especially in combination with student data to explore retention-related issues for first-time-in-college students. The dataset being created will be freely available, and others are encouraged to add their own plans of study. We offer the standard operating procedures in this paper, along with data conventions, to best facilitate the large-scale analysis of this type of network data. As the dataset grows, we anticipate the ability of the community to understand and interrogate the programmatic barriers to student success in engineering across the nation will also expand – leading to a cornucopia of previously unexplored questions at scale.