2023 ASEE Annual Conference & Exposition

Multidisciplinary Authoring – A Critical Foundation for Augmented Reality Systems in Training and Education

Presented at Multidisciplinary Engineering Division (MULTI) Technical Session 3

Recent advances in Augmented Reality (AR) devices and their maturity as a technology offers new modalities for interaction between learners and their learning environments. Such capabilities are particularly important for learning that involves hands-on activities where there is a compelling need to: (a) make connections between knowledge-elements that have been taught at different times, (b) apply principles and theoretical knowledge in a concrete experimental setting, (c) understand the limitations of what can be studied via models and via experiments, (d) cope with increasing shortages in teaching-support staff and instructional material at the intersection of disciplines, and (e) improve student engagement in their learning.

AR devices that are integrated into training and education systems can be effectively used to deliver just-in-time informatics to augment physical workspaces and learning environments with virtual artifacts. We present a system that demonstrates a solution to a critical registration problem and enables a multi-disciplinary team to develop the pedagogical content without the need for extensive coding. The most popular approach for developing AR applications is to develop a game using a standard game engine such as UNITY or UNREAL. These engines offer a powerful environment for developing a large variety of games and an exhaustive library of digital assets. In contrast, the framework we offer supports a limited range of human- environment interactions that are suitable and effective for training and education. Our system offers four important capabilities – annotation, navigation, guidance, and operator safety. These capabilities are presented and described in detail.

The above framework motivates a change of focus – from game development to AR content development. While game development is an intensive activity that involves extensive programming, AR content development is a multi-disciplinary activity that requires contributions from a large team of graphics designers, content creators, domain experts, pedagogy experts, and learning evaluators. We have demonstrated that such a multi-disciplinary team of experts working with our framework can use popular content creation tools to design and develop the virtual artifacts required for the AR system. These artifacts can be archived in a standard relational database and hosted on robust cloud-based backend systems for scale up. The AR content creators can own their content and Non-fungible Tokens to sequence the presentations either to improve pedagogical novelty or to personalize the learning.

Authors

Emi Aoki University of Massachusetts, Lowell [biography]

Emi Aoki is a PhD student in Electrical Engineering at the University of Massachusetts Lowell. Her past research experience has been in data analysis and augmented reality application. Her research interests include stochastic modeling and machine learning.
Bach Tran University of Akron [biography]

Bach Tran completed his Ph.D. degree in Computer Engineering at the University of Akron, Ohio, USA in 2022. His research focuses on Software Systems designing, including AI, AR, and IoT Systems. His most recent work involves designing an AR framework that provides just-in-time informatics for training, education and maintenance activities. This system design was the basis for a Future of Work planning grant from NSF in 2021. He has been leading the team at the University of Akron for the NASA SUITS competition from 2018 to 2020. He currently works as a software developer.
Nathan Esosa Uhunsere University of Massachusetts, Lowell [biography]

Member of the Centers for Advanced Computational and Telecommunications Research
Dr. Susan Thomson Tripathy http://orcid.org/0000-0001-5359-6045 University of Massachusetts, Lowell [biography]

Dr. Susan Thomson Tripathy is a Research Associate at UMass Lowell who specializes in qualitative research methods. She received her doctorate in Anthropology from Harvard University.
Prof. Charles Thompson, Ph.D. University of Massachusetts, Lowell [biography]

Charles Thompson is Professor of Electrical and Computer Engineering, Director of the Center for Advanced Computation and Telecommunications and formerly Associate to the Dean for Research and Graduate Study at the University of Massachusetts at Lowell. He received a B.S. in Electrical Engineering from New York University, a M.S. in Electrical Engineering from the Polytechnic Institute of NewYork, and a Ph.D. in Acoustics from the Massachusetts Institute of Technology.

Upon graduation he became an Assistant Professor of Engineering Science and Mechanics at the Virginia Polytechnic Institute and State University. In 1987 he joined the Department Electrical and Computer Engineering at UMASS Lowell as its Analog Devices Career Development Professor. Dr. Thompson has served on the executive boards of the Cooperative Research Fellowship program of Bell Laboratories (1991-1999) and the AT&T Labs Fellowship Program (1996-2006).

At Bell Laboratories Dr. Thompson created with the Vice President of Research and Nobel laureate, Arno Penizas, the W. Lincoln Hawkins Mentoring Excellence Award (1994). This award is given to a member of the research staff for fostering the career growth of Bell Labs students and associates. This award is Research‚Äôs highest honor for mentoring contributions. In 1998, AT&T Labs instituted a similar award named for Dr. Thompson.

Charles Thompson is Professor of Electrical and Computer Engineering, Director of the Center for Advanced Computation and Telecommunications and formerly Associate to the Dean for Research and Graduate Study at the University of Massachusetts at Lowell. He received a B.S. in Electrical Engineering from New York University, a M.S. in Electrical Engineering from the Polytechnic Institute of NewYork, and a Ph.D. in Acoustics from the Massachusetts Institute of Technology.

Upon graduation he became an Assistant Professor of Engineering Science and Mechanics at the Virginia Polytechnic Institute and State University. In 1987 he joined the Department Electrical and Computer Engineering at UMASS Lowell as its Analog Devices Career Development Professor. Dr. Thompson has served on the executive boards of the Cooperative Research Fellowship program of Bell Laboratories (1991-1999) and the AT&T Labs Fellowship Program (1996-2006).

At Bell Laboratories Dr. Thompson created with the Vice President of Research and Nobel laureate, Arno Penizas, the W. Lincoln Hawkins Mentoring Excellence Award (1994). This award is given to a member of the research staff for fostering the career growth of Bell Labs students and associates. This award is Research‚Äôs highest honor for mentoring contributions. In 1998, AT&T Labs instituted a similar award named for Dr. Thompson.

His awards include the US Presidential Award for Excellence in Mentoring; Tau Beta Pi Eminent Engineer; James E. Blackwell Scholar; AT&T Bell Laboratories Cooperative Research Fellowship. He is cited in Who‚Äôs Who among African Americans, Education, and Technology Today; American Men and Women of Science, West Babylon Alumni Hall of Fame; He is a Fellow of the Acoustical Society of America and cited for his fundamental contributions to theoretical and computational acoustics. He is senior member of IEEE, and a member of the American Physical Society and Sigma Xi. He has published reesearch in acoustics, control theory, fluid mechanics, heat transfer, linear and nonlinear systems, and telecommunications.
Prof. Shivakumar Sastry http://orcid.org/0000-0002-9236-851X University of Akron [biography]

Prof. Shivakumar Sastry is a Professor with the Department of Electrical and Computer Engineering, The University of Akron. He received his Ph.D. degree in Computer Engineering and Science from Case Western Reserve University and holds Masters Degrees in Computer Science from University of Central Florida and in Electrical Engineering from the Indian Institute of Science. His research interests are in Networked Embedded Systems, Real-time systems, Graph algorithms and Network Analysis. Prior to joining Akron, he was a Senior Research Scientist with Rockwell Automation. He is currently also serving as the Director of Strategic Initiatives.
Prof. Kavitha Chandra University of Massachusetts, Lowell [biography]

Kavitha Chandra is the Associate Dean for Undergraduate Affairs and Professor of Electrical and Computer Engineering in the Francis College of Engineering at the University of Massachusetts Lowell. She directs the Research, Academics and Mentoring Pathways (RAMP) to Success summer bridge and academic program for new engineering students, preparing them with research, communication and leadership skills. Her research interests are in computational and data-driven modeling of physical systems in acoustics and communication networks, model-based systems engineering, user-centric design of emerging technology and engineering education.

Download paper (4.82 MB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session

For those interested in:

Academia-Industry Connections
Broadening Participation in Engineering and Engineering Technology