2024 ASEE Annual Conference & Exposition

Understanding how engineers connect technical work to broader social-ecological systems is critical because their designs transform societies and environments. As part of a national study to explore how civil and chemical engineers navigate design decisions, we are developing a survey instrument to assess mental models of social-ecological-technical systems (SETS). Mental models (Johnson-Laird, 2001; Rouse & Morris, 1986), are internal representations that individuals use to describe, explain, and predict the form, function, state, and purpose of a system. In this case, the system is the connection between technical design and broader social-ecological systems. The project is informed by three frameworks: 1) planned behavior, 2) mental models, and 3) social-ecological-technical systems (SETS). The project integrates the theory of planned behavior with mental models to build fundamental knowledge of engineers’ mental models of SETSs, changes in their mental models over time, and relationships between mental models and design decisions.

This paper presents the instrument development process centered on eliciting mental models of SETS. SETS (McPhearson et al., 2022) is a generalized framework that positions social, technical, and ecological elements of a system as vertices of a triangle, with interactions in all directions. The instrument will include both closed-ended and open-ended items, allowing us to leverage advances in natural language processing to scale qualitative data analysis and combine an inferential framework often associated with quantitative studies with the richer information flow associated with qualitative studies.

Previous work using SETS has identified individual components within each vertex salient to the specific context (Bixler et al., 2019). In this paper, we report on the phases of instrument development that support this contextualization: 1) Initial interview protocol development followed by semi-structured interviews with six engineering students outside the target majors to test how well the protocol elicits information about students mental models of SETS, 2) revisions to the interview protocol followed by semi-structured interviews with senior-level students in chemical and civil engineering students (12 per discipline), 3) deductive and inductive analysis of those interviews, using SETS as our deductive coding scheme followed by inductive coding to refine and contextualize the analysis and support survey development. We conclude with the initial survey instrument, which will undergo pilot testing in the summer of 2024. The results both support instrument development and offer an exploratory analysis of civil and chemical engineering students’ mental models of SETS.

Bixler, R. P., Lieberknecht, K., Leite, F., Felkner, J., Oden, M., Richter, S. M., ... & Thomas, R. (2019). An observatory framework for metropolitan change: Understanding urban social–ecological–technical systems in Texas and beyond. Sustainability, 11(13), 3611.

Johnson-Laird, P. N. (2001). Mental models and deduction. Trends in cognitive sciences, 5(10), 434-442.

McPhearson, T., Cook, E. M., Berbes-Blazquez, M., Cheng, C., Grimm, N. B., Andersson, E., ... & Troxler, T. G. (2022). A social-ecological-technological systems framework for urban ecosystem services. One Earth, 5(5), 505-518.

Rouse, W. B., & Morris, N. M. (1986). On looking into the black box: Prospects and limits in the search for mental models. Psychological bulletin, 100(3), 349.