2023 ASEE Annual Conference & Exposition
Context Matters: Continued Study of Results of Common Concept Questions at Several Diverse Institutions
Presented at Will This Be on the Mechanics Test? Concept Inventories and Understanding Exams
Concept-based instruction is an approach to deploy “concept questions” which are qualitative and designed to elicit patterns of thought that complement or reinforce those required for procedural questions. Typically, concept questions are multiple choice with one “correct” answer among several “attractive distractors”. However, some concept questions may, by design, have “multiple defensible responses”, so as to engender debate and deeper discussion about multiple solution pathways, underlying assumptions, or other contextual details. Also, the use of concept questions is arguably most effective when written explanations of answers are also collected, so as to better understand students’ reasoning, including the possibility that an “incorrect” answer reveals some measure of conceptual understanding (sometimes referred to as a “phenomenological primitive”). Finally, use of concept questions is part of an evolutionary process of faculty development, in which the deployment, review of explanations, and feedback, is an ongoing process oriented toward effective teaching and learning outcomes.
A Community of Practice (CoP) of mechanics instructors from several diverse institutions (ranging in size, demographics, and identity), has been formed to use the Concept Warehouse (CW) as a platform to create, deploy, and assess the results of concept questions in Statics and Dynamics. The CW is an online tool that contains several thousand concept questions, called “ConcepTests”, that range over several topics in engineering, including approximately 800 in mechanics. The CW allows the instructor to deploy the ConcepTests in a variety of modalities, including online or offline, in-class or out of class, and with response time allocated to be “immediate” (say 2-5 minutes during class) or “extended” (say several hours or days as a preparatory or exploratory exercise).
The CoP has two teams, one for Statics, and one for Dynamics. During the 2022-23 academic year, each member of each team will assign the same four “common questions” from the CW, at the point and in the modality appropriate to their course. The following data will be collected from students: the answer to the question, corresponding written explanations (i.e., to explain or justify the chosen answer), and feedback (e.g., confidence and impressions as to the usefulness of the question). Some students and faculty will be selected for follow-up interviews.
This work is the sequel to a work-in-progress (WIP) article published and presented at the 2022 Annual Conference & Exposition, that was conducted by four faculty teaching Statics. The four common statics questions in this study are the same as used in the WIP. The WIP reported two general findings:
(i) across all institutions, and independently of correctness of their answers, female students consistently reported lower confidence in their answers; (ii) among students selecting correct responses, only about one third to one half expressed reasoning that was considered “correct”. Nevertheless, many “incorrect” answers contained portions of reasoning that suggested that some core ideas were being expressed, allowing for the possibility of further discussion to build understanding.
This study will add further data, including information from student and faculty interviews that was not available for the WIP, to validate or cross-examine these results. In particular, the notion that scores of a concept question are an imperfect measure of students' conceptual understanding will be further explored. The paper will also discuss if there is evidence to suggest similarities or differences results across the institutional contexts.
-
Dr. Christopher Papadopoulos University of Puerto Rico, Mayaguez [biography]
-
Dr. Jean Carlos Batista Abreu
http://orcid.org/0000-0001-5380-7682
Elizabethtown College
[biography]
-
Dr. Brian P. Self California Polytechnic State University, San Luis Obispo [biography]
-
Dr. Anna K. T. Howardhttp://orcid.org/0000-0003-0207-6757
North Carolina State University at Raleigh
[biography]
Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.