2024 ASEE Annual Conference & Exposition

This investigation is the first of four investigations funded by the NSF (DUE award 2215807) to develop and then field test on open educational browser-based writing-to-learn tool called GIKS. The underlying theory is that writing-to-learn with immediate formative feedback presented as concept networks is engaging and effective for learning concepts covered in lectures. This work was studied in a second year architectural engineering course focusing on building materials, processes and modeling. Participants (n=84) completed a lesson (readings, lecture, and labs) then followed by writing prompts centered on the following topics: Building with Concrete and Wood Construction (3 weeks later). Participants were assigned to one of two counterbalanced groups, group A used GIKS software to write a 300-word summary of the first lesson but did not write in the second lesson, while group B did not write in the first lesson but used GIKS in the second lesson, so that each group served as a control treatment for the other group. All students completed a concept structure survey at the end of each lesson that contained 20 key concepts from that lesson, the two concept structure surveys’ data were transformed into concept networks and then these networks were compared to an expert network benchmark referent, as well as to networks of the textbook chapter and the PowerPoint slides of the related lesson. Then a week after the second lesson students completed the standing end-of-module multiple-choice posttest that included items from these lesson as well as from other lessons in the module.
Results to date highlight that for both lessons, the group using GIKS scored higher on the concept structure survey (more like the expert network) BUT lower on the multiple-choice test, the difference was significant for the Building with Concrete lesson (p < .05) but not for the Wood Construction lesson. This interaction has been reported before by Ntshalintshali & Clariana (2020), that improving conceptual knowledge sometimes decreases memory of lesson details. Descriptive analysis of the group-average networks derived from the concept structure surveys for Building with Concrete show that the group-averaged network of those using GIKS compared to the control was more like the expert network (54% vs. 36%), the network of the textbook Chapter (32% vs. 29%), the network of the PowerPoint (PP) (46% vs. 41%), and especially like peers in the other group (67%). For Wood Construction the difference between the groups was less, the group-averaged network of those using GIKS compared to the control was more like the expert (40% vs. 39%), like the light-framed construction PP (28% vs. 24%), and especially like peers in the other group (72%). These findings show that writing-to-learn with GIKS with immediate network feedback improves conceptual knowledge as expected but at the cost of details. Peers conceptual structure of the lesson materials were very similar (peer-peer mental model convergence) and were more like others than like the expert, or the book chapters, or the PowerPoint slides; in addition, the PowerPoint slides appear to influence conceptual structure more than the textbook chapters. Investigation 2 will consider writing-to-learn with or without immediate network feedback in order to isolate the effects of immediate network feedback.