Papers on AR-Mediated Human-Robot Collaboration Coming Out in RO-MAN and MobileHCI

Two AR-mediated human-robot collaboration papers we have been working on for quite some time are coming out later this summer. 

fARfetch: Enabling Collocated AR-HRC in Large Visually Diverse Environments with VLM-Driven AR Content Adaptation is about human-robot collaborations in large, natural environments. Using a Meta Quest 3 and a Unitree Go2 robot dog, we demonstrate a system that combines a semantically aware world-in-miniature AR representation of the environment with vision-language-model-driven content adaptation. Compared with a baseline approach, the proposed method enables faster task completion and reduces user workload across several measures.

ARTOO-DARTU: Studying AR-HRC With AR Obstruction Mitigation During a Warehouse Task is about mitigation of AR content obstruction of important elements of the real world. In this work, for which we build an AR-mediated HRC system with a Microsoft HoloLens 2 and a TurtleBot 4, we demonstrate a scenario where AR assistance is appreciated only when AR-induced obstruction of the real world is detected and mitigated. We have been developing methods for detecting AR-induced obstruction of the real world in a parallel line of work (see VIDDAR, TVCG 2025) — it is very nice to also demonstrate that obstruction detection and mitigation can have a positive impact on user experience, in the important case of AR-mediated HRC.

Both studies are led by Christian Fronk, who is very happy to get a chance to present the outcomes of his work in Wales, UK and Kitakyushu, Japan, all in a course of two last weeks of August. Way to go Christian!

About Maria Gorlatova

Dr. Maria Gorlatova is the Sternberg Family Associate Professor of Electrical and Computer Engineering at Duke University. Her research centers on pervasive computing technologies, with a focus on spatial computing and next-generation augmented reality. She received her Ph.D. in Electrical Engineering from Columbia University and her M.Sc. and B.Sc. degrees (Summa Cum Laude) in Electrical Engineering from the University of Ottawa, Canada. She spent two years at Princeton University’s Electrical Engineering Department as an Associate Research Scholar and Associate Director of the Princeton University EDGE Lab. Dr. Gorlatova was named one of the N2Women Rising Stars and has received multiple awards, including the NSF CAREER Award, the DARPA Young Faculty Award (YFA), the DARPA Director’s Fellowship, the IEEE ISMAR 2025 Best Paper Award, the IEEE Communications Society Award for Advances in Communications, and research awards from Meta, Cisco, and IBM. She served as the TPC Co-chair of ACM/IEEE IPSN 2023, the General Co-Chair of the 2025 NSF Workshop on Networking and Systems Challenges in Immersive Computing, and the TPC Co-chair of ACM/IEEE SenSys 2025.
This entry was posted in Augmented reality, Publications and Demonstrations, Robotics, Safety. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *