2023 ASEE Annual Conference & Exposition

Introduction:
This work-in-progress study assesses the impact of reflective practices, including peer review, on written assignments in a sophomore-level, biomedical engineering laboratory course. The course serves as an introduction to experimentation and covers statistical design of experiments as well as how to quantify the quality of experimental measurement data. Topics include problem-solving skills, scientific writing, hypothesis generation, amongst other research-related topics. Course pedagogy includes standards-based grading and reflection.

Our previous work identified weaknesses in the “problem identification” and “interpretation” components of problem solving [1]. As a result, we implemented the evidence-based strategy of reflective practices [2] and noticed a trend suggesting that increased reflective practices, namely the addition of reflective engineering notebooks, may have improved student perception of standards-based grading as well as may increase their engagement with mastery of course standards [3]. To address limitations of our previous study and build upon encouraging results, we will implement the reflective, equitable strategy of peer review [4] on individual written assignments. We hypothesize that the implementation of peer review on individually written extended abstracts will result in increased mastery in course standards, namely “problem identification” and “interpretation”, and result in favorable student attitude with regards to the peer review process.

Methods: In-class:
Written assignments will be assessed by peer reviewers and also by the instructional team, using an enhanced Likert-based, problem-solving rubric [5] which expands on “problem identification” and “interpretation”. Both students and the instructional team will undergo grader calibration. During the peer review process, peer reviewers will only provide written feedback (no Likert scores) to follow the evidenced-based process of ungrading [6].

Methods: Study Analysis:
Methodologies used in the study will include quantitative and qualitative survey data analysis and analysis of written lab assignments. More specifically, the study team will grade the draft and final submissions using the Likert scale and providing comments. Only the grades for the final submission will be released; however, the researchers will analyze pre- and post-submissions to assess the impact of the peer review process on standards mastery. Further, the researchers will grade the quality of the peer reviews and investigate correlations with mastery levels of both the critics and critiqued. Lastly, the study team will investigate student attitude using surveys.

Results:
The study team has developed two instruments for assessment: 1) the assessment of peer review quality and 2) an attitudinal survey regarding the peer review process. More specifically, the peer review quality instrument (based on [7]) will assess the appropriateness and specificity of the criticism as well as the justification for why it is included. Lastly, each review will be assessed for the presence and usefulness of any suggestions. The attitudinal survey (based on [8]-[10]) contains two sections, one for the critic and one for the critiqued, and covers areas such as utility, assessment of training/grader calibration, impact on future work, and emotion.

Discussion and Conclusion:
The use of reflective practices improves the implementation of standards-based grading [2]. Our work has seen a trend suggesting improved engagement and standard-based grading perception [3]; however, further work is needed to assess without previous study limitations. More specifically, this work-in-progress assesses the implementation of an additional reflective practice (peer review) with the goal of improved individual mastery in problem solving and written communication while retaining high student favor.

References:
[1] blinded
[2] H. A. Diefes-Dux and A. R. Carberry, “Cases of Student Reflection within a Course Using Standards-Based Grading,” in 2019 IEEE Frontiers in Education Conference (FIE), 2019, pp. 1–9. doi: 10.1109/FIE43999.2019.9028501.
[3] blinded
[4] Feldman, J. (2019). Grading for equity: What it is, why it matters, and how it can transform schools and classrooms.
[5] blinded
[6] Kohn, A., & Blum, S.D. (2020). Ungrading: Why Rating Students Undermines Learning (and What to Do Instead). (First edition. ed.). Morgantown: West Virginia University Press.
[7] S. Gielen, E. Peeters, F. Dochy, P. Onghena, and K. Struyven, “Improving the effectiveness of peer feedback for learning,” Learning and Instruction, vol. 20, no. 4, pp. 304–315, Aug. 2010, doi: 10.1016/j.learninstruc.2009.08.007.
[8] Andersson, Magnus, and Maria Weurlander. “Peer Review of Laboratory Reports for Engineering Students.” European journal of engineering education. 44.3 (2019): 417–428. Web.
[9] Conde, Miguel A. “Application of Peer Review Techniques in Engineering Education.” The International journal of engineering education. 33.2 (2017): 918–926. Print.
[10] Larsen, Katarina, and Johan Gärdebo. “Retooling Engineering for Social Justice: The Use of Explicit Models for Analytical Thinking, Critical Reflection, and Peer-Review in Swedish Engineering Education.” International journal of engineering, social justice and peace. 5.1-2 (2018): 13–29. Web.