2024 ASEE Annual Conference & Exposition

Work in Progress: Investigating the Impact of College Students’ Personal Characteristics on Peer Assessment: A Multilevel Linear Modeling Approach

Keywords: Peer Assessment; Multilevel Linear Modeling; Personal Characteristics

Abstract: Teamwork is an essential skill expected of college students, and it has been frequently integrated into courses by instructors, making the assessment of these activities an important component of university curricula. Peer assessment serves as a crucial assessment method, offering valuable feedback and enhancing student learning outcomes. It is commonly used in college courses that employ team-based learning, and there is a growing focus on the impact of peer assessment design on student learning outcomes. Previous research highlights correlations between peer ratings and factors related to personal characteristics, including gender, academic performance, personality traits, and group preferences; however, a significant gap remains in our understanding of how these factors specifically influence peer ratings in the context of college course teamwork. This knowledge gap hinders our ability to provide instructors, students, and researchers with evidence-based insights into the effectiveness of team-based pedagogy and into the interpretation of peer assessments. To address this gap, our study employs multilevel linear modeling to investigate the relationships between these factors and peer rating scores in the context of college course teamwork.

We collected peer rating data from 5,322 college students at a Midwest research university spanning the period from 2019 to 2023. The data were collected using Tandem, a digital instructional tool designed to foster equitable teamwork. We employed a four-level linear model where responses nested in the crossing of students and items, which in turn are nested in teams within courses. Peer ratings measured on a 9-Likert scale serve as the dependent variable, and the main factors include gender, cumulative GPA, personality traits (Extraversion and Task Control), and group preference. Our analysis revealed statistically significant influences of these factors. Students with higher cumulative GPAs earned greater average peer rating values compared to their peers. Students who rated themselves higher in extraversion and preference for task control received lower ratings, on average. In addition, on average, students who indicated a preference for working in groups received lower peer assessment scores than those who preferred individual work or collaboration.

Our results emphasize the importance of considering the effects of gender, group preference, and personality traits in the design of peer assessment for evaluating team-based learning outcomes. Moreover, when conducting research aimed at examining team-based learning outcomes and related factors (e.g., designed interventions), we should also account for the effects of personal factors that are not usually considered, such as gender, academic performance, personality traits, and group preferences.