2023 ASEE Annual Conference & Exposition

This is a method paper. The social relations model (SRM) is a general conceptual and methodological framework to depict voluntary or involuntary interpersonal relationships and interactions between two or more individuals within groups. SRM is commonly used in education and psychology contexts to examine interpersonal behaviors across group members, and it has wide applicability in engineering education research given the growing emphasis on team learning and collaborative project-based learning in the field.
There is an abundance of longitudinal team learning data in engineering education. For example, the Comprehensive Assessment of Team Member Effectiveness (CATME) system has been used by over 1.4 million students and 17,000 instructors, and it is prevalent among engineering instructors. Engineering instructors often collect multiple rounds of round-robin peer evaluation and assessment data within student teams across multiple periods during a semester, and sometimes across multiple semesters. Previous studies in engineering education and other fields have applied the standard SRM to longitudinal social relations data. However, the standard SRM is developed based on cross-section data, and studies have shown that it has critical limitations when applied to longitudinal data, where we observe the same individuals across multiple periods.
Motivated by the lack of available methods in analyzing the Longitudinal Social Relations Model (LSRM), in this paper, we use a Bayesian approach to design a general and flexible framework to bridge the gap. With a simulation-based framework, we show that the Bayesian method has significant advantages over two alternatives (a two-step method and the Social Relations Structural Equation Model) in terms of estimation accuracy. The Bayesian method has other advantages as well. It is straightforward to incorporate covariates, e.g., demographic variables, into the Bayesian LSRM, and it handles missing data very well, which is quite common among student round-robin peer-evaluation datasets. The Bayesian LSRM opens the pathway to analyze a series of potentially impactful and policy-relevant questions in engineering education, for example, the drifts of peer evaluation accuracy among students across time and the dynamic impacts of cultural diversity on students’ collaborations.