2024 ASEE Annual Conference & Exposition

State of the art curriculum development efforts today are generally undertaken by solely faculty members of the program. However, in our previous work [anonymized citation], we showed how expert crowdsourcing combined with the application of a consensus building method can be used to perform curriculum development asynchronously with a larger group of experts beyond a program’s faculty. The consensus building method included two operations by the expert crowd: (1) validating the existing list of curriculum topics and their subtopics; and (2) suggesting additional topics and subtopics to be added to the current curriculum. This paper will show results yielded by a finalized experiment utilizing consensus building method against a graduate technology management course’s curriculum development.

This paper will then detail how this research effort incorporated a professional diversity factor into the consensus building method when performing expert crowdsourcing. Professional diversity is important because when building consensus among the experts, we also want to ensure there are enough representatives from various relevant categories of professionals. To calculate professional diversity, we performed a five-step process: (1) Identify the relevant factors, (2) Identify the interactions between factors and relevant categories to consider, (3) Identify the relevance of the identified categories, (4) Identify the ideal recruitment needs, and (5) Compute a global professional diversity measure based on current recruitment. Relevant factors represent statistical variables for the expert crowd population which may include years of domain experience (e.g., junior, senior), the industry they’re in (e.g., industry, government, or education), and the position they hold (e.g., executive, manager, practitioner). Identifying the interaction between the different factors and which combinations will also be considered (if we have independent variables or dependent variables). For instance, the input of a junior engineer in industry versus an assistant professor in academia. Both are juniors but in different industries. One solution is to consider them as diversity categories: junior versus senior, or industry versus academia. Another solution is to consider the categories: junior in industry and junior in academia. If the factors are independent with respect to the subject matter, then keeping simple distinct categories will be easier. However, we may determine that the views of a junior in industry are quite different from the views of a junior in academia, and therefore decide to keep the combined categories (this will lead to more categories but also more accurate results). In this case the factors will be dependent. Such an analysis will be performed both at the beginning using domain expertise but also during the experiment analyzing the answers received and adapting prior decisions.

While this paper is a work in progress, the experiment currently running to test the professional diversity factor will be completed by the end of 2023 – so the results, analysis, and discussions will be available for the draft and final paper submissions.