2025 ASEE Annual Conference & Exposition

The Engineering Professional Skills Assessment 2.0: Preparing Engineering Students for Global Workplace Complexities

This is an abstract proposing an Evidence-Based Practice, Research Brief paper for consideration in the ASEE 2025 Educational Research and Methods Division conference proceedings.

Proficiency in professional skills such as collaboration, knowledge application in contemporary contexts, ethical judgment, problem solving, and capacity for continued learning are among those identified by employers as necessary for success in the 21st century global work environment. Engineering program accrediting bodies worldwide recognize this importance and require evidence of student mastery of related student outcomes for almost a quarter century. Yet, faculty in engineering programs struggle to define, teach and measure these professional skills in their efforts to generate accurate and useful data for course and program-level assessment purposes.

The Engineering Professional Skills Assessment (EPSA) is the only direct method in the literature that can be used to teach and measure engineering student performance of five learning outcomes simultaneously. The EPSA is a discussion-based performance assessment with two components: (1) a task in the form of a prompt and a scenario that presents a contemporary multi-faceted engineering problem in a complex societal and environmental context with no clear-cut solution, and (2) a task-specific analytic scoring rubric designed to be used to evaluate the student discussion in response to a prompt and scenario.

A study of the reliability and validity of the EPSA was funded by the US National Science Foundation (NSF DUE #: 1432997). This paper presents previously unpublished (1) inter-rater reliability results of 191 scores of student discussion transcripts by the study’s engineering faculty using the EPSA Rubric; (2) updates to the EPSA Rubric that increase its clarity and relevancy while maintaining previously established instrument validity; and (3) recommendations for using the EPSA for course and program level assessment, as well as future research.

The target used for acceptable interrater reliability is 70% prior to rater team calibration (Stemler, 2004). Five engineering faculty used the EPSA Rubric to score the same set of discussion transcripts, producing 70 scores and an inter-rater reliability percentage of 79.4%. After discussing the results of the first set of transcripts using consensus estimate approach calibration guidelines (ETS, 2007), the same five faculty formed rater pairs and scored a second set of transcripts, producing 121 scores and an inter-rater reliability percentage of 91.6%. Details regarding the data, processes and procedures used to establish interrater reliability will be in the paper.

The original EPSA rubric had been tied directly to six of the ABET EAC Criterion 3 Student Outcomes as published in 2000. The updated rubric is accreditation organization agnostic; the wording of the definitions and some of the descriptors were modified to increase clarity and ensure relevancy and flexibility of use for programs worldwide, not matter the programmatic accrediting body. Revisions were informed by use of the rubric, follow accepted rubric development and evolution guidelines. The revision team ensured that the original instrument validity was upheld. The table below compares the 2016 EPSA Rubric with the 2024 EPSA Rubric learning outcomes. A full discussion of the revision choices and a mapping of outcomes to global accrediting bodies, including ABET’s EAC and ANSAC student outcomes, will be detailed in the paper.

2016 EPSA Rubric
1. Understanding of professional and ethical responsibility.
2. Broad understanding of the impact of engineering solutions in global, economic, environmental, and cultural/societal contexts.
3. Ability to communicate effectively.
4. Recognition of the need for and ability to engage in life-long learning.
5. Knowledge of contemporary issues.

2024 EPSA Rubric
1. Students problem solve in an ethical manner.
2. Students consider impacts of solutions on relevant contexts.
3. Students communicate with each other to reach consensus.
4. Students acquire, interpret, evaluate and apply information.
5. Students consider contemporary issues.

Authors

Dr. Ashley Ater Kranov Washington State University [biography]

Dr. Ashley Ater Kranov is an adjunct associate professor in the School of Electrical Engineering and Computer Science at Washington State University.
Dr. Edwin R. Schmeckpeper P.E. Norwich University [biography]

Edwin Schmeckpeper, P.E., Ph.D., was formerly Professor and Chair of the Department of Civil Engineering and Construction Management at Norwich University, which is recognized by ASCE as the first private school in the United States to offer engineering courses. Norwich University was the model used by Senator Justin Morrill for the land-grant colleges created by the 1862 Morrill Land Grant Act. Prior to joining the faculty at Norwich University, Dr. Schmeckpeper taught at a land-grant college, the University of Idaho, and worked as an engineer in design offices and at construction sites.
Dr. Steven W. Beyerlein University of Idaho [biography]

Dr. Beyerlein joined the University of Idaho in 1987 and worked as a Mechanical Engineering faculty member for 35 years until he retired in 2022. He facilitated the creation and growth of the University of Idaho multi-disciplinary capstone design program beginning in 1996 and served as a department chair from 2015-2020. He is recognized an ASME Sparks Medal for his design and delivery of interactive learning experiences. His research has been in applied thermodynamics as well as scholarship of teaching/learning where he is widely cited for his work on process-oriented learning, assessment for learning, program assessment, and faculty development.

Note

The full paper will be available to logged in and registered conference attendees once the conference starts on June 22, 2025, and to all visitors after the conference ends on June 25, 2025