Changing the World, One Day at a Time

Two AR-mediated human-robot collaboration papers we have been working on for quite some time are coming out later this summer. 

fARfetch: Enabling Collocated AR-HRC in Large Visually Diverse Environments with VLM-Driven AR Content Adaptation is about human-robot collaborations in large, natural environments. Using a Meta Quest 3 and a Unitree Go2 robot dog, we demonstrate a system that combines a semantically aware world-in-miniature AR representation of the environment with vision-language-model-driven content adaptation. Compared with a baseline approach, the proposed method enables faster task completion and reduces user workload across several measures.

ARTOO-DARTU: Studying AR-HRC With AR Obstruction Mitigation During a Warehouse Task is about mitigation of AR content obstruction of important elements of the real world. In this work, for which we build an AR-mediated HRC system with a Microsoft HoloLens 2 and a TurtleBot 4, we demonstrate a scenario where AR assistance is appreciated only when AR-induced obstruction of the real world is detected and mitigated. We have been developing methods for detecting AR-induced obstruction of the real world in a parallel line of work (see VIDDAR, TVCG 2025) — it is very nice to also demonstrate that obstruction detection and mitigation can have a positive impact on user experience, in the important case of AR-mediated HRC.

Both studies are led by Christian Fronk, who is very happy to get a chance to present the outcomes of his work in Wales, UK and Kitakyushu, Japan, all in a course of two last weeks of August. Way to go Christian!