2026 ASEE Annual Conference & Exposition

Background. The Standards for Computer Science (CS) Teachers by the Computer Science Teachers Association (CSTA) articulate indicators of effective classroom practice. To assist teachers in understanding their proficiency and areas for growth, we designed and pilot-tested a vignette-based assessment of middle school (6th-8th grade) CS teacher proficiencies aligned with Standards 2, 4, and 5.

Research Question. Our research questions for this study was: How difficult did teachers find the new measure? and How did the teachers perform on the vignettes?

Methodology. We constructed three vignettes with questions aligned to Standards 2, 4, and 5 and geared toward middle school teachers. Our cognitive interviews with middle school teachers (n = 5) informed iterative revisions of the assessment prior to pilot testing with middle school teachers. We then revised the assessment and piloted it with additional teachers (n = 19).

Results. At the scale level, teachers performed moderately (M = 58.56%, SD = 15.09%, Min = 20.00%, Max = 90.00%). The items discriminated fairly well between higher- and lower-performing test takers overall, indicating that performance on each item was generally correlated with performance on all other items in the assessment. The pilot revealed that three items needed attention since they were not highly correlated with the performance of the other items.

Implications. This emerging measure of teacher proficiencies is expected to demonstrate promising psychometric properties, though additional revisions to items may be warranted following pilot testing. Once validated, the measure could serve as a useful tool for practitioners to identify areas of strength and opportunities for professional growth, informing future professional development efforts.