IEEE/ACM IPSN Paper: Edge-assisted Collaborative Image Recognition for Mobile Augmented Reality

Posted on May 18, 2020 by Maria Gorlatova

Our paper on image recognition for mobile augmented reality in the presence of image distortions appeared in IEEE/ACM IPSN’20 and received the conference’s Best Research Artifact Award [ Paper PDF ] [ Presentation slides ] [ Video of the presentation ]

CollabAR: System Architecture

This work is motivated by the unavoidable presence of image distortions, such as motion blur and Gaussian noise, in practical mobile augmented reality systems. A large body of work that deals with mobile-captured images assumes that distorted images can be filtered out. However, our study of realistic augmented reality scenarios demonstrated the pervasiveness of distortions: for example, a user wearing a Magic Leap One headset in an indoor corridor in our engineering building would have 99.6% of the images captured by a Magic Leap One camera corrupted by motion blur. We thus developed a solution, CollabAR, whose architecture is shown above. CollabAR runs on an edge server. CollabAR (1) analyses images to detect whether they are corrupted by noise, (2) uses a collection of “experts,” optimized to perform well with specific types of noise, to perform image recognition, (3) takes advantages of the potential presence of temporally and spatially related images to further improve recognition results. CollabAR achieves over 96% recognition accuracy for images that contain severe multiple distortions, while achieving the end-to-end system latency as low as 17.8 ms.

Examples of distorted images in the MVMDD dataset we collected as part of this work.

Our IEEE/ACM IPSN paper describing CollabAR is accompanied by an open-source repository that includes the examples of real-world distortions we observed in realistic mobile augmented reality systems, the new multi-view dataset we have collected, the code we developed to generate distortions, and the source code for the core pieces of the CollabAR framework.

This work is supported in part by NSF grants CSR-1903136 and CNS-1908051 and by the Lord Foundation of North Carolina.

About Maria Gorlatova

Dr. Maria Gorlatova is the Sternberg Family Associate Professor of Electrical and Computer Engineering at Duke University. Her research centers on pervasive computing technologies, with a focus on spatial computing and next-generation augmented reality. She received her Ph.D. in Electrical Engineering from Columbia University and her M.Sc. and B.Sc. degrees (Summa Cum Laude) in Electrical Engineering from the University of Ottawa, Canada. She spent two years at Princeton University’s Electrical Engineering Department as an Associate Research Scholar and Associate Director of the Princeton University EDGE Lab. Dr. Gorlatova was named one of the N2Women Rising Stars and has received multiple awards, including the NSF CAREER Award, the DARPA Young Faculty Award (YFA), the DARPA Director’s Fellowship, the IEEE ISMAR 2025 Best Paper Award, the IEEE Communications Society Award for Advances in Communications, and research awards from Meta, Cisco, and IBM. She served as the TPC Co-chair of ACM/IEEE IPSN 2023, the General Co-Chair of the 2025 NSF Workshop on Networking and Systems Challenges in Immersive Computing, and the TPC Co-chair of ACM/IEEE SenSys 2025.
This entry was posted in Achievement, Augmented reality, Edge computing, Funding and Awards, Mobile computing, Publications and Demonstrations. Bookmark the permalink.