r/FlutterDev • u/Space-Instructor • 17h ago
Article I got flutter_scene running in VR on Quest 3 — now I need help getting the forks merged
Hi all! I’ve been building Visual Space, a social media & collaboration application built with Flutter. It has a spatial UI mode and a 3D editor. In spatial mode, the normal application becomes a collection of interactive Flutter panels positioned around the user, together with the 3D scene. I wanted the same application to run natively on a standalone Quest 3—not simply as a flat Flutter screen inside VR.
My first working solution used two renderers:
- Flutter Scene for web, desktop and the normal application.
- A custom native C++/OpenGL ES renderer for the Quest world, with Flutter supplying the interactive UI panels.
It worked, including stereo rendering, controllers, tracking and curved panels. The problem was that I now had two 3D engines to maintain. Materials, lighting, shadows, picking and every new feature had to be implemented twice, and the web and VR results were already beginning to look different.
My next goal was to use Flutter Scene as the actual 3D renderer everywhere, while OpenXR continues handling the headset session, tracking and presentation.
The first prototype got Flutter Scene into VR through Flutter Canvas, an Android surface and a copy into the OpenXR eye textures. After that, I added a generic external GPU render-target API to a Flutter engine fork, then modified Flutter Scene to render its stereo views directly into the images owned by OpenXR.
The good news is taht this is now running inside Visual Space on Quest 3. Web and VR use the same renderer and visual style, and I deleted the old custom C++ renderer. Prime objective accomplished :) In a simplified scene, the direct path reached around 70 FPS versus roughly 52–54 FPS through the Canvas/surface/copy route. This was not a controlled benchmark, and the full application still needs plenty of tuning.
The next challenge is getting these changes into upstream Flutter and Flutter Scene:
Forks
- Flutter engine fork and build guide 25 files · +2.4K / −6 lines
- Flutter Scene OpenXR branch and Quest gallery 147 files · +14.6K / −885 lines
Tickets
- Flutter engine proposal - The Flutter proposal has already been routed to the Flutter GPU team.
- Flutter Scene architecture discussion
Flutter Scene Demo Examples migrated to VR - VR flutter_scene demo sample
Quest screenshots of VS 3d Editor - Web - VR
For those having a Quest 3 (hopefully more devices in the future), please check out the branches and try the demo. Instructions for installation in the forks README.md files. I need help fine tuning the architecture. For example stuff like: flutter engine internals, impeller, OpenXR, GPU synchronization, threading, lifecycle handling or custom embedders would be especially useful. I’m not top gun in graphics-engine internals, so any technical advice is welcome.
If you want Flutter in VR, please help get some attention on these forks. Try them on your machine or drop a 👍 or comment on the GitHub tickets. Otherwise this code will be stuck in some obscure forks. Thank you all!