2026 ASEE Annual Conference & Exposition

Engineering and Health Science Collaborative Model for Assessing Augmented Reality and Vision-Based Hand-Tracking Systems

Presented at Multidisciplinary Engineering Division (MULTI) Technical Session 8: Collaborative and Team Learning, and Assessment

The potential of digital health solutions with immersive technology that can respond to the growing demand for remote healthcare, such as physical therapy, motivates the need for educational frameworks that prepare engineering students to be better informed about designing healthcare applications. The focus is on Augmented reality (AR) and computer-vision-based hand tracking systems, both of which offer promising solutions for remote rehabilitation by enabling real-time capture of hand motion and gesture data; however, they differ in sensing modality, accuracy, latency, cost, and implementation complexity. This work presents an interdisciplinary educational framework that integrates clinically relevant information from health sciences to inform the design and assessment of technology based applications for hand tracking. The modules are designed to be accessible to undergraduate students in engineering and health-sciences, promoting increased collaboration between these disciplines. The modular framework enables students to implement, analyze, and compare 3D sensor-fusion-based AR tracking with 2D computer-vision-based tracking, focusing on clinically relevant metrics such as joint angles, gesture classification, and range-of-motion estimation. Two rehabilitation-inspired, project-driven activities-gesture-based object manipulation in a spatial grid and robotic arm control-provide hands-on experiential learning aligned with physical therapy concepts and human-computer interaction principles. The educational content is derived from ongoing joint research by graduate students in engineering and physical therapy, with licensed physical therapists guiding the process. By embedding immersive technologies within a structured, outcome-based instructional model that is also open-access, this work demonstrates a scalable approach to teaching the design of digital health technologies that incorporate engineering, rehabilitation, and human-centered design.

Authors
  1. Gayathri Boopathy University of Massachusetts Lowell [biography]
  2. Emi Aoki University of Massachusetts Lowell [biography]
  3. Flore Stécie Norcéide University of Massachusetts Lowell [biography]
  4. Ravi Kkumaar Sivasankar Venkata University of Massachusetts Lowell [biography]
  5. Dr. Erika S. Lewis University of Massachusetts Lowell [biography]
  6. Prof. Charles Thompson PhD University of Massachusetts Lowell [biography]
  7. Prof. Kavitha Chandra University of Massachusetts Lowell [biography]
Note

The full paper will be available to logged in and registered conference attendees once the conference starts on June 21, 2026, and to all visitors after the conference ends on June 24, 2026

« View session

For those interested in:

  • 1st Generation
  • 2 Year Institution
  • Academia-Industry Connections
  • Advocacy and Policy
  • Broadening Participation in Engineering and Engineering Technology
  • computer science
  • disability
  • engineering
  • engineering technology
  • Faculty
  • gender
  • Graduate
  • information technology
  • LGBTQIA+
  • New Members
  • Pre-College
  • professional
  • race/ethnicity
  • Socio-Economic Status
  • transfer
  • undergraduate
  • veterans