2024 ASEE Annual Conference & Exposition

Tactile Learning: Making a Computer Vision Course Accessible through Touched-Based Interfaces

Presented at Transformative Learning in STEM: Accessibility, Social Impact, and Inclusivity in Higher Education

The term "visual learner" is a ubiquitous concept in education. It is often associated with experiential- or example-based teaching that helps the students understand a concept through its application. However, for those with visual impairments, visual learning may not be an option. The Royal National Institute for the Blind reports that there are significant barriers to learning technical content for blind individuals, including access to visual resources and, correspondingly, difficulty interpreting visual concepts. In this work, we discuss experiences and key takeaways from adapting aspects of a computer science graduate course to be more accessible for the blind. The course teaches advanced machine learning methods rooted in computer vision and gesture-based methods in order to build sketch recognition systems. Additionally, it emphasizes elements of human-computer interaction and interface design, many of which are visual concepts. In order to adapt the curriculum, we used a high-resolution tactile display capable of mirroring imagery from a video display into a depth map that could be felt. This enabled the dual presentation of visual content as tactile surface maps. Through this process, we learned several best practices in terms of how to create content that transfers well from one modality to another, and we also developed a number of guidelines for creation of teaching materials like notes and assignments in a way that is more screen-reader friendly. This paper shares key takeaways while also communicating student and teacher perspectives on developing, teaching, and using these materials with the goal of encouraging more technical courses that are traditionally visually-based to consider possible ways to becoming more accessible.

Authors

Dr. Seth Polsley http://orcid.org/0000-0002-8805-8375 University of Nebraska, Lincoln [biography]

Dr. Seth Polsley is an Assistant Professor of Practice in the Jeffrey S. Raikes School of Computer Science and Management at the University of Nebraska–Lincoln, with his academic home in the School of Computing. His research focuses on the combination of intelligent systems design and human-computer interaction in order to support novel educational and universal computing experiences.
Ms. Amanda Kate Lacy [biography]

Amanda Lacy is a PhD student at Texas A&M University in the department of Computer Science and Engineering. Her interests are broad, with an emphasis on applying computing to promote access to information and spaces, both virtual and physical. She hol
Samantha Ray http://orcid.org/0000-0003-3189-8899 Texas A&M University [biography]

Samantha Ray is a Computer Engineering PhD student at Texas A&M University. Her research focuses on creating intelligent systems for tasks that require human-like levels of understanding. She has previously worked on human activity recognition (HAR) syste
Dr. Tracy Anne Hammond http://orcid.org/https://0000-0001-7272-0507 Texas A&M University [biography]

Dr. Hammond is Director of the Texas A&M University Institute for Engineering Education & Innovation and also the chair of the Engineering Education Faculty. She is also Director of the Sketch Recognition Lab and Professor in the Department of Computer Science & Engineering. She is a member of the Center for Population and Aging, the Center for Remote Health Technologies & Systems as well as the Institute for Data Science. Hammond is a PI for over 13 million in funded research, from NSF, DARPA, Google, Microsoft, and others. Hammond holds a Ph.D. in Computer Science and FTO (Finance Technology Option) from the Massachusetts Institute of Technology, and four degrees from Columbia University: an M.S in Anthropology, an M.S. in Computer Science, a B.A. in Mathematics, and a B.S. in Applied Mathematics and Physics. Hammond advised 17 UG theses, 29 MS theses, and 10 Ph.D. dissertations. Hammond is the 2020 recipient of the TEES Faculty Fellows Award and the 2011 recipient of the Charles H. Barclay, Jr. '45 Faculty Fellow Award. Hammond has been featured on the Discovery Channel and other news sources. Hammond is dedicated to diversity and equity, which is reflected in her publications, research, teaching, service, and mentoring. More at http://srl.tamu.edu and http://ieei.tamu.edu.

Download paper (1.76 MB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session