2025 ASEE Annual Conference & Exposition

Automated Analysis of Knowledge Types in Computer Science Textbooks: A Natural Language Processing Approach to Understanding Epistemic Climate

Presented at Computers in Education Division (COED) Track 6.D

This study investigates how computer science textbooks present different forms of knowledge to students, contributing to the epistemic climate that shapes students' understanding of what counts as valid knowledge in their field. While curricular materials significantly influence students' epistemic development, traditional textbook analysis methods are limited by their labor-intensive nature and discipline-specific applicability. To address these methodological limitations, we developed a framework conceptualizing engineering knowledge as distinct "knowledge types" with unique purposes, linguistic characteristics, and epistemic implications. Through an analysis of computer science textbooks and leveraging natural language processing techniques, we identified ten distinct knowledge types ranging from conceptual and mathematical to ethical and metacognitive. We generated a synthetic dataset of 10,000 labeled textbook passages using large language models and demonstrated the feasibility of training transformer-based models to automatically classify knowledge types, achieving F1-scores between 0.68-0.97 across different categories. The results revealed significant co-occurrence patterns between knowledge types and varying model performance, suggesting both the interconnected nature of engineering knowledge and opportunities for more intentional knowledge representation in educational materials. This approach offers a scalable method for analyzing epistemic climate in engineering education, with implications for curriculum development, textbook design, and understanding how knowledge representation influences students' disciplinary identity formation.

Authors
  1. Mitchell Gerhardt Orcid 16x16http://orcid.org/0009-0006-4191-1654 Virginia Polytechnic Institute and State University [biography]
  2. Dr. Andrew Katz Virginia Polytechnic Institute and State University [biography]
Note

The full paper will be available to logged in and registered conference attendees once the conference starts on June 22, 2025, and to all visitors after the conference ends on June 25, 2025