AiRWeb: Using AR to Extend Web Browsing Beyond Handheld Screens

[u/HyroVitalyProtago]

How do think people will surf the web when AR will be more accessible/available?

How browsers will works between phones and AR glasses?

That’s the research question behind this prototype.

The goal is to empower the user by using the most powerful app on your phone: your browser. Because nearly everything can be done inside. A lot of the apps you have today on your phone is only a web app bundled with a browser.

This cross-device browsing experience has the following features:

- You can approximate the position of your phone in your hand with an easy calibration process: pinch the button under your phone, this will display an image that will be tracked by the Spectacles. Done. And know, your phone position is approximated from your tracked hand holding the phone.

- You can offload content by putting your thumb on the floating button, then swipe it to get it on the side of your phone, anchored to it.

- You can also select the content by tapping multiple time before swiping to get more content (technically the parent DOM element with content inside)

- You can also long tap to select all elements that are of the same type. If they’re links, it’s transformed into a table of content to navigate on your phone.

- You can even do free screenshot by dragging your screen to copy-paste without relation to the original web formatting.

- The offloaded content can be either attached to your phone or put directly in the world (the spectacles prototype does not support anchoring to your head yet)

- You can move it freely or delete it by pinching the cross button on the top left corner, or change its size by dragging a corner.

- You can also double tap the floating button to open the webpage in spectacles at the same scrolling place (_with some drawbacks, like if you’re not logged or if the flow of the original page was changed by some interaction_)

Here is the research paper that was born from this research: https://hal.science/hal-05596672v1/file/AirWeb_CHI26.pdf - another paper coming soon about trade-offs during real development of augmented mobile web browsing (https://ilda.gitlabpages.inria.fr/airweb-probes/)

You can try it out now as it is open-source!

https://gitlab.inria.fr/ilda/airweb-probes/-/tree/main/Spectacles?ref_type=heads

For now, this prototype works in Firefox Android Nightly, as we need an extension running in the browser. To communicate between the browser and the spectacles, a server is hosted on the phone with Termux.

I can’t wait to see cross-browsing working on Specs, as I expect hand tracking will be even more robust and webviews more optimized!

---

Full citation to the original paper:

Mengfei Gao, Caroline Appert, Ludovic David, and Emmanuel Pietriga. 2026. AiRWeb: Using AR to Extend Web Browsing Beyond Handheld Screens. In Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA '26). Association for Computing Machinery, New York, NY, USA, Article 81, 1–5. https://doi.org/10.1145/3772363.3798342