2023 ASEE Annual Conference & Exposition

This Great Ideas For Teaching Students (GIFTS) paper will describe evaluation of a first-year advising program first implemented in academic year 2021-22 in the engineering school at a medium-sized private university – [program name redacted] (authors redacted, 2022 ASEE Annual Conference). Our [program name redacted] program builds on the Advising-as-Teaching learner-centered approach to advising at Northwestern University’s McCormick School of Engineering and Applied Science (Freeman, Gentry, and Goldberg, 2016 ASEE Annual Conference), and similarly aims to leverage a naturally developing community within our first-year engineering design course, [first-year course number redacted]. While many of our [program advisor titles redacted] are involved with our first-year design course as either an instructor or a design team technical mentor, this is not universally true; some of our [program advisor titles redacted] are not involved in our first-year course. This is a distinction between our [program name redacted] program and Northwestern’s Advising-as-Teaching model in which the first-year advisors and the first-year engineering program instructors are one and the same.

The goal of [program name redacted] is to provide whole-student support for our first-year students through holistic mentorship that extends beyond curricular questions and purely academic concerns. [Program name redacted] is designed to also support students should they encounter life challenges as they transition to university life. It additionally aims to provide broader mentoring as first-year students begin discovering their own paths to lifelong personal fulfillment.

The evaluation of [program name redacted] focuses on 3 direct questions and 1 indirect question. The direct questions are:
1) Is the advising provided by [program name redacted] providing what students want from advising – is [program name redacted] perceived by students as meeting students’ goals?
2) Is the advising provided by [program name redacted] effective in supporting students’ development of self-efficacy for self-regulated learning, engineering self-efficacy, and resilience – is [program name redacted] working?
3) Is the advising provided by [program name redacted] dependent on the [program advisor title redacted]’s involvement in the first-year engineering design course (instructor, technical mentor, or no formal involvement) – are [program advisor names redacted] not involved in [first-year course number redacted] just as effective as advisors as [program advisor names redacted] who are involved in [first-year course number redacted]?
The indirect question repurposes aggregated institutional data regarding engineering program retention into the second year:
4) Does the advising provided by [program name redacted] correlate with retention in the engineering program – do we see a change in students’ intent to continuing pursuing engineering that appeared to take place concurrently with the introduction of the new first-year advising model (engineering population overall, as well as among sub-groups with historically lower engineering retention – women, BIPOC students)?

We are evaluating the efficacy of our [program name redacted] program by eliciting student perspectives through selected questions from four validated survey instruments: a survey of 12 essential advising functions proposed by Smith and Allen, 6 of the original 11 statements in a survey of self-efficacy for self-regulated learning proposed by Zimmerman, et al., a 6-statement general engineering self-efficacy survey proposed by Mamaril, et al., and a 6-statement resiliency survey proposed by Smith, et al., that focuses on resilience as, “the ability to bounce back or recover,” from challenges. The survey, currently under review by our Institutional Review Board, asks students to rate their agreement with the survey statements when they think back to themselves at the start of their first year (I.e., a post-pre self-assessment) and when they think about themselves now, as fall semester sophomores (post self-assessment). A limitation of this evaluation is the lack of a control group of first-year students who were not advised through [program name redacted], so we are unable to disambiguate growth that may be attributed to [program name redacted] from growth that may be attributed to completing the first year of an engineering program. In addition to evaluating student self-assessment of the degree of students’ perceived growth in their engineering self-efficacy, self-efficacy for self-regulated learning, and resilience during their first year, we will also evaluate historical aggregated institutional data regarding engineering school retention to see if there appears to be a change that took place concurrently with the implementation of [program name redacted]. We will continue to evaluate trends in the coming years to more fully evaluate the potential impact of [program name redacted] on engineering program retention.

Authors redacted. (2022) Work-in-Progress: Paper title redacted. ASEE’s 129th Annual Conference and Exposition. Minneapolis, MN June 26—29, 2022.
R. T. Freeman, K. Gentry, and J. E. Goldberg (2016) Changing the Advising Model. ASEE’s 123rd Annual Conference and Exposition. New Orleans, LA June 26—29, 2016.
C. L. Smith and J. M. Allen, “Essential Functions of Academic Advising: What Students Want and What Students Get,” NACADA Journal, vol. 26, no. 1, pp. 56-66, 2006.
B. J. Zimmerman, A. Bandura and M. Martinez-Pons, “Self-Motivation for Academic Attainment: The Role of Self-Efficacy Beliefs and Personal Goal Setting,” American Educational Research Journal, vol. 29, no. 3, pp. 663-676, 1992.
N. A. Mamaril, E. L. Usher, C. R. Li, D. R. Economy, and M. S. Kennedy, “Measuring Undergraduate Students’ Engineering Self-Efficacy: A Validation Study,” Journal of Engineering Education, vol. 105, no. 2, pp. 366-395, 2016.
B. W. Smith, J. Dalen, K. Wiggens, E. Tooley, P. Christopher, and J. Bernard, “The Brief Resilience Scale: Assessing the Ability to Bounce Back,” International Journal of Behavioral Medicine, vol. 15, no. 3, pp. 194—200, 2008.