$400K to transform design education with augmented reality

Reading time: 2 minutes

A multidisciplinary team of professors at University of Hawaiʻi at Mānoa was awarded a $400,000 grant to develop an augmented reality learning system for design education. Led by architecture, computer science and education faculty the three-year project aims to create a more immersive and collaborative learning experience for undergraduate students.

“This grant will allow us to address a critical skills shortage in the architecture, engineering and construction industry sector using innovative technologies—at the same time, it provides a research focus and funding to support emerging faculty and graduate research assistants,” said Principal Investigator Eric Peterson from the School of Architecture.

diagram — As groups of students engage with a physical model in the immersive learning environment, their movements are tracked by the motion capture system and processed by an AI learning agent. In response, information is displayed on top of the model.

The new learning system will incorporate advanced technologies such as Spatial Augmented Reality (SAR), Tangible User Interfaces (TUI) and Natural Language Processing. SAR projects images onto surfaces without requiring headsets. TUI uses physical models that interact with a computer, allowing students to engage with physical models in a realistic learning environment, while their movements are tracked by a motion capture system and analyzed by an AI learning system.

“My teaching has always centered on helping students develop structural intuition through hands-on experiments, iteration, and feedback,” said School of Architecture Professor Yasushi Ishida, who is also co-principal investigator. “I’m excited to carry this approach further through interdisciplinary collaboration, using new technologies to create immersive and collaborative ways for students to engage with structural design.”

Impacting STEM teaching

The project will also have a broader impact on how STEM subjects are taught. The team believes their findings will reveal optimal student learning methods, which can be applied to other subjects where models or diagrams aid in understanding complex concepts.

“In the last few years, my collaborators and I have worked on communicating scientific data of geo-spatial nature by combining tangible interfaces and spatial augmentation,” said Nurit Kirshenbaum from the Information and Computer Sciences Department, who is also co-principal investigator. “It is exciting to bring our research to this interdisciplinary project where we can focus on education in architecture and engineering as a new application domain for our technology.”

The project team also includes Seungoh Paek and Daniel Hoffman from the College of Education’s Department of Learning Design & Technology, who will bring their expertise in developing and testing technology-enhanced learning.

“This interdisciplinary project is a fantastic opportunity to bring together expertise from across units in a way that will benefit learners and make UH Mānoa shine,” said Hoffman.

link