The Hill I Will Die On: AI Smart Glasses Are A Useful Tool, Not An Interface To Our Entire Lives

The hill I will die on: we will need smart glasses for work, and for helping us with well-defined, specific tasks, long before we will need smart glasses worn continuously, all day every day.

Humans already use eyewear extensively as specialized equipment. Loupes. Safety goggles. Welding glasses. Reading glasses when needed. The leap from putting on reading glasses to knit to putting on smart glasses to repair a coffeemaker is not large. Smart glasses + AI are an immediate asset here, offering clear and unquestionable value (once the remaining AI kinks are worked out): they can help people complete tasks faster, make fewer mistakes, and stay safer. Likewise, glasses are an immediate aid for humans working with robots: a robot’s internal state is largely invisible to the human supervising it; a head-mounted interface can display that state, show intentions and warnings, and make human oversight far more efficient. Smart glasses as specialized gear are easy to understand and easy to justify.

All-day smart glasses though – that is a whole other animal. There is little precedent, aside from assistive and medical devices, in relying on a device to help you all the time, all day every day. In certain circumstances, people may desire continuous support: e.g., when trying to change entrenched behavior, or feeling that their own resources are overwhelmed. These are important use cases, with real potential to improve people’s lives. We wrote about using smart glasses for supporting personal development and change before, and I am as excited to work towards realizing this important vision as ever.

AI smart glasses have the potential to be an aid for people who seek development and change. More in our 2022 position paper.

Continue reading

Posted in Augmented reality, Augmented Reality For Good, Perspectives, Technology | Leave a comment

Do We Need New Exam Rules for the Smartglasses Era?

It is definitely happening: students are using smartglasses to cheat on their exams.

Anecdotal evidence of this has been reported on Reddit for quite some time: for example, this week and  two months ago. It is clearly technically feasible to use smart glasses to help on an exam, given how easy it is to connect glasses that have a camera in them to capable back-end AI models. There are useful documented demonstrations of this: for example, a system based on Rokid + ChatGPT 5.2 was demonstrated to get a student to a score of 92.5 on a computer networking exam for which the class average was 72. There are instances of cheating for which criminal penalties may be appropriate: for example, in Korea, a person taking a fire engineer exam is facing criminal prosecution for cheating using AI-powered glasses.

Continue reading

Posted in How the world is changing, Perspectives, Publications and Demonstrations, Technology | Leave a comment

AR and Wearable AI Need to be Trustworthy to be Useful

A lovely article in The Economist this week: AI agents lie, cheat and steal. That is putting off users. The article makes a point that is perhaps obvious: AI needs to be reliable to be useful. E.g., omitting the second-highest result when asked for the top 10 — the example used in the article — is bad when answers matter. The article argues for the importance of the “trust layer” for the AI, to facilitate its adoption in business-critical use cases.

This argument translates directly to wearable AI, i.e., smart glasses, and augmented reality as a whole. We accept mistakes, like visuals that are incorrect or misplaced, in research prototypes and in games — and even there, only up to a point. Continue reading

Posted in Perspectives, Technology | Leave a comment

Papers on AR-Mediated Human-Robot Collaboration Coming Out in RO-MAN and MobileHCI

Two AR-mediated human-robot collaboration papers we have been working on for quite some time are coming out later this summer. 

fARfetch: Enabling Collocated AR-HRC in Large Visually Diverse Environments with VLM-Driven AR Content Adaptation is about human-robot collaborations in large, natural environments. Using a Meta Quest 3 and a Unitree Go2 robot dog, we demonstrate a system that combines a semantically aware world-in-miniature AR representation of the environment with vision-language-model-driven content adaptation. Compared with a baseline approach, the proposed method enables faster task completion and reduces user workload across several measures.

ARTOO-DARTU: Studying AR-HRC With AR Obstruction Mitigation During a Warehouse Task is about mitigation of AR content obstruction of important elements of the real world. In this work, for which we build an AR-mediated HRC system with a Microsoft HoloLens 2 and a TurtleBot 4, we demonstrate a scenario where AR assistance is appreciated only when AR-induced obstruction of the real world is detected and mitigated. We have been developing methods for detecting AR-induced obstruction of the real world in a parallel line of work (see VIDDAR, TVCG 2025) — it is very nice to also demonstrate that obstruction detection and mitigation can have a positive impact on user experience, in the important case of AR-mediated HRC.

Both studies are led by Christian Fronk, who is very happy to get a chance to present the outcomes of his work in Wales, UK and Kitakyushu, Japan, all in a course of two last weeks of August. Way to go Christian!

Posted in Augmented reality, Publications and Demonstrations, Robotics, Safety | Leave a comment

New Funding: 4-year NSF CISE FutureCore Award on Automated AR Guidance Monitoring

We are grateful to the NSF for supporting our work on automated AR guidance monitoring, in pervasive AR and human-robot collaboration (HRC) applications. This award will support our joint work with Miroslav Pajic’s group at Duke, which is very exciting to me personally as this synergetic collaboration will help ensure impact on multiple research communities.

AR guidance can be hard to read, appear in the wrong place, provide incorrect instructions, hide important real-world information, or encourage actions that are unsafe, intrusive, or poorly suited to the surrounding environment. A user may not always notice these problems before acting on the guidance. This project will develop new ways to automatically check whether augmented reality guidance is clear, correct, safe, and appropriate in the physical setting where it appears.

Posted in Augmented reality, Edge computing, Funding and Awards, Highlights, Mobile computing, Research, Robotics, Safety, Security | Leave a comment

RIP Magic Leap Headsets

It is official: Magic Leap is no longer making its own headsets. Our thoughts are with everyone affected by the layoffs. May you land on your feet; any team will be lucky to have you.

Magic Leap has a very special place in the XR research ecosystem: this headset became a true workhorse for XR research labs using eye tracking to study attention and spatial interaction. Its compact form factor made it uniquely practical for experiments involving mobile cognition and substantial head-and-body movement, such as our lab’s work with AR-assisted CPRContinue reading

Posted in Augmented reality, Perspectives, Technology | Leave a comment

Keynote and Technology Demonstrations at IEEE VisArch Co-located with ISCA

Duke I3T Lab at IEEE VisArch Workshop co-located with IEEE ISCA

It was our pleasure to participate in IEEE VisArch Workshop co-located with IEEE ISCA, which came to Raleigh, NC in June 2026.

Professor Gorlatova delivered a keynote speech titled “Intelligent and Secure XR: Building Adaptive Spatial Systems That Know When They Are Wrong;” PhD students Yanming Xiu and Christian Fronk brought their live technology demonstrations previously showcased as part of DARPA Intrinsic Cognitive Security Program PI meetings.

Posted in Augmented reality, Edge computing, Events, Publications and Demonstrations, Research, Students and Mentorship, Talks, Technology | Leave a comment

Recent I3T Lab Graduate Sarah Eom Highlighted by the Duke University Center for Computational and Digital Health Innovation

Sarah Eom, PhD, was highlighted by the Duke University Center for Computational and Digital Health Innovation for her work at the intersection of next-generation immersive systems and healthcare. Congratulations Dr. Eom!

Posted in Highlights, Research, Students and Mentorship | Leave a comment

Is This the Future for Smart Glasses and AI?

If you do not believe that the future is smartglasses + AI, the VisionClaw paper might change your mind.

The authors created a system that “running on Meta RayBan smart glasses, continuously perceives real-world context and enables in-situ, speech-driven action delegation via Gemini Live and OpenClaw AI agents.” Paper’s description of a (small; N=4) deployment study is fascinating in its demonstration of how agentic interaction changes everyday behavior.

I’ve always believed in augmented reality – obviously! – but this is just a whole other level of opportunities for AR and AI.

Full text of the paper on ArXiv 

Posted in Perspectives | Leave a comment

I3T Lab Tour a Part of Regional NAE Meeting

It was our sincere pleasure to offer a Duke I3T Lab Tour as part of the National Academy of Engineers Regional Meeting held at Duke University in May 2026.

 

May 2026 NAE Regional Meeting attendees exploring live AR and VR demonstrations as part of the Duke University Pratt School of Engineering lab tour

Just like the Frontiers of Engineering (FOE) NAE Symposium I attended back in 2021, the Regional Meeting was a real treat: it outlined both the promise and the challenges in the exciting and important space of commercial aviation, from multiple technical and non-technical perspectives. Like the FOE Symposium, it reminded me how exciting it is to be an engineer: to have not only an in-depth understanding of the problems and the underlying processes, but also know how to solve them. Thankful for the organizers for the opportunity to participate in this event and to contribute to it. [ Short video with several snippets from the lab tour ]

 

Posted in Augmented reality, Demonstrations, Duke University, Events, Highlights, Lab tours, Mobile computing, Students and Mentorship | Leave a comment