2025 ASEE Annual Conference & Exposition

This full theory paper describes how engineering education researchers can employ the Delphi method to explore threshold concepts in interdisciplinary subfields of engineering. Threshold concepts are pivotal ideas that, once mastered, fundamentally transform a learner's thinking and sense of identity within a discipline. In engineering, this shift is often referred to as "thinking like an engineer" and "feeling like an engineer.” Although this way of thinking about fundamental knowledge in a field was proposed just over two decades ago, the process for identifying a threshold concept is still challenging. Of the various features defining threshold concepts are considered to be “bounded” – delineating a conceptual space that establishes boundaries between disciplines. The idea of a concept being “bounded” poses a salient issue for interdisciplinary fields compounding the other difficulties: how can we identify potential threshold concepts in these spaces? Among the methods proposed in the literature, the Delphi method is perhaps one of the more promising approaches for developing a set of potential threshold concepts. Our core contributions in this paper will involve (1) reviewing previous efforts using the Delphi method to identify threshold concepts across disciplines and (2) outlining our approach to the Delphi method using the interdisciplinary field of cyber-physical systems (CPS) in contrast to previous studies.

The Delphi method is a consensus-building technique where a panel of experts responds to multiple rounds of questionnaires, refining their feedback until consensus is reached. This method has been applied in engineering education settings to develop the engineering education taxonomy of keywords, a classification scheme for first year engineering courses, and, of particular relevance to this paper, proposing threshold concepts. In our study, we involved 11 experts in CPS and interdisciplinary engineering, including smart grids, autonomous systems, and machine learning, in three rounds of surveys. We achieved a 100% response rate across all rounds.

Unlike other Delphi studies used to identify threshold concepts, we focused on identifying “core” concepts in CPS and used questions to probe panelists about whether the “core” concepts exhibited any of the threshold concept qualities. In round two, we asked panelists to rate their agreement on the core concepts and rank them based on the extent to which they aligned threshold concepts qualities. In the subsequent round, we encouraged participants to further frame the concepts in terms of ways of thinking to align more cohesively to the premise threshold concepts as transformative using established approaches for Delphi studies. This was done to limit the generation of broad concepts (e.g., frequency domain, differential equations) that are unusable from a curricular (re)design perspective.

By combining these approaches, we contend this approach to a Delphi study offers a framework for identifying potential threshold concepts in interdisciplinary subfields of engineering. To maximize the usability of this work, we will share the prompts and analytical details for engineering education researchers to adopt in their own efforts.