AdaptSLAM, our recent work led by Ying Chen, explores new approaches for adapting edge computing-supported Visual and Visual-Inertial Simultaneous Localization and Mapping (V- and VI-SLAM) to computation and communication resource constraints.
Adapt SLAM’s system architecture. Our design centers on the two highlighted modules. We optimize our algorithms to run in real time on mobile devices.
Sarah Eom has demonstrated our ongoing work on AR-based magnification for hand-held loupes at ACM SenSys’22 in Boston, MA. This work is based on a collaboration with Miroslav Pajic (Duke ECE) and Majda Hadziahmetovic (Duke Ophthalmology). [Demo description PDF] [Accompanying poster PDF]
A poster we presented at ACM UbiComp 2022 in Cambridge, UK, titled “IoT-Enabled Environment Illuminance Optimization for Augmented Reality“, received the ACM UbiComp’22 Best Poster Award. [ Poster submission ] [ Poster presented at the conference ]
This poster outlines a system that uses a smart lightbulb, an edge server, and environmental sensors (a camera and a light sensor) to change the level of light in the environment to maximize the performance of two elements of augmented reality, pose tracking and eye tracking. To our knowledge, this is the first automatic environment optimization system for augmented reality that adapts to both environment lighting and textures.
We are delighted to have two of lab’s papers appear at the top AR/MR conference, IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2022 (acceptance rate: 21%).
In a paper led by PhD student Sarah Eom titled NeuroLens: Augmented Reality (AR)-based Contextual Guidance through Surgical Tool Tracking in Neurosurgery, we developed a system that provides neurosurgeons with real-time guidance on how to approach a target located inside a patient’s skull. This system, developed in collaboration with Dr. Shervin Rahimpour, was evaluated in a study with 33 medical students, who conducted both AR-guided and unassisted (freehand) trials of catheter insertion into a phantom model of a human head. The study has demonstrated that our system significantly improves students’ targeting accuracy. The study has also revealed important differences in the behavior of participants who achieved different levels of results in AR-assisted settings. More than 93% of the participants agreed or strongly agreed that the developed system is useful for learning to conduct neurosurgical procedures. [ Paper PDF ] Continue reading
At the Athena AI Institute‘s 1st Annual Summit held at Duke University in August 2022, 4 members of the lab presented posters and showcased the work of the lab during the I3T Lab Tour.
The projects showcased during the summit covered different elements of next-generation edge computing-supported augmented reality (AR): edge-supported resource-efficient SLAM for AR, edge-coordinated IoT for improving the performance of AR, context-aware AR for neurosurgery, and robust object detection for AR.
Jeremy Suh, Grand Challenges in Engineering NSF REU Fellow at the Pratt School of Engineering, who has spent his summer in the I^3T Lab, has presented a poster and a demo of his work at the Duke University summer REU showcase.
Jeremy with his mentors, Tim Scargill and Lin Duan.
Jeremy’s research centered on showcasing how edge computing-supported object detection can be used as part of augmented reality (AR) applications in foreign language learning. The framework Jeremy has created will be used as a foundation for a range of semantically aware AR applications that are being developed in the lab.
This week Ying Chen is presenting our work on evaluating and exploiting characteristics of user pose in VR at IEEE INFOCOM’22, in a paper and an accompanying demonstration:
VR Viewport Pose Model for Quantifying and Exploiting Frame Correlations presents the first statistical model of viewport pose in VR and develops the first analytically grounded algorithm that establishes which contents should be reused across the frames [PDF] [Code and data]
Demo: Pixel Similarity-Based Content Reuse in Edge-Assisted Virtual Reality showcases how adaptive cross-frame content reuse reduces bandwidth requirements in edge computing-supported VR [Demo abstract PDF] [Video of the demo]
We are presenting at 3 CPS-IoT Week sessions this week:
IPSN’22: EyeSyn: Psychology-inspired Eye Movement Synthesis for Gaze-based Activity Recognition presents the first method for synthesizing eye movement data for training eye movement-based activity classifiers for AR and VR without human involvement. [PDF] [Code and data] [NSF Discoveries news item covering this work]
Heartfelt congratulations to Alex Hu for receiving the Spring 2022 ECE Independent Study Best Poster Award, for his poster titled “Impact of the Environment on Eye Tracking Efficacy in Headset Augmented Reality“. The poster was prepared as part of Alex’s year-long Duke ECE undergraduate Honors Thesis study titled “Eye Tracking for User and Environment-Aware Headset Augmented Reality“.
We work in the area of pervasive mobile and sensing systems broadly, and pervasive mobile Augmented Reality (AR) and next-generation intelligence for the Internet of Things in particular. Our work is generously supported by the National Science Foundation (NSF), the Lord Foundation of North Carolina, IBM, Facebook, and DARPA. We are also a part of the NSF AI Institute for Edge Computing Leveraging Next Generation Networks, where our work is focused on building next-generation AI-powered mobile augmented reality.
If you are interested in joining the lab as a postdoc, please e-mail professor Gorlatova (maria.gorlatova /at/ duke.edu) your CV, a brief note about your research interests, and 1-3 papers that you believe represent your best work to date. Candidates for postdoctoral positions should have a track record of publishing their work in top venues of the field, and need to be self-driven and independent. The postdoctoral position start dates are flexible, and can be as early as May 2022 and as late as January 2023. Lab’s previous postdoctoral affiliate Dr. Guohao Lan has successfully secured an independent Assistant Professor position at a top university.
Additional information:
