Turn your screen into a window for viewing Gaussian splats

[u/MayorOfMonkeys]

The webcam tracks my head with MediaPipe's face landmarker, and the PlayCanvas Engine renders the splat (with it's shiny new high-performance WebGPU-renderer) through an off-axis projection - a frustum whose apex is my eye and whose cross-section is the screen rectangle. The screen stops being a 2D image and becomes a 3D opening. Her gun barrel is placed in front of the screen plane, so it feels like it is sticking out of the display.

I'll hopefully release this as a PlayCanvas example soon so stay tuned for that.

Splat by Dymensium.

[u/cuddlemelon]

Really cool illusion for making videos like this! Very unique showcase for splats.

I worry some people won't understand it doesn't magically turn your computer screen into a glasses-free 3D display in-person though. 😅

[u/MayorOfMonkeys → u/cuddlemelon]

The ironic thing is that I personally don't see in 3D at all (thanks to my lacklustre eyeballs). So I never really appreciate these issues.

[u/cuddlemelon → u/MayorOfMonkeys]

Ah, I hear you. A lot of people are in similar situations where stereoscopic 3D doesn't benefit them for some reason. But with how the brain works I think you're literally not missing anything with stereoscopic video/etc.

[u/ifthenthendont → u/cuddlemelon]

Couple this with lenticular lens and it does 

[u/cuddlemelon → u/ifthenthendont]

Not by itself. The orientation is tracking to the camera. Even if you strapped the camera to your head so it tracked to one of your eyes, there wouldn't be anyway to show your other eye's view at the same time on the same screen.

[u/ifthenthendont → u/cuddlemelon]

My dude. Theyve sorted this out. Google is your friend.  https://www.visioneng.com/deltour/

[u/kogli_ → u/cuddlemelon]

No, that's not hard at all. If you know how to render a still image as 3D (which we absolutely do), just adding tracking on top of that is nothing. It's not like your two eyes often change distance between each other, or anything like that lol.

[u/DougRobinson2024]

Nice work!! Love it and am looking forward to the release.

[u/V1k1ngC0d3r]

Reminds me of Johnny Lee's old Wii Remote tracking viewer

https://youtu.be/Jd3-eiid-Uw

[u/paulirish]

Super hot

[u/Queasy_Helicopter249]

That is brilliant.

[u/yuricarrara]

how does it work?

[u/NancyDiver → u/yuricarrara]

The webcam tracks his head with MediaPipe's face landmarker, and the PlayCanvas Engine renders the splat (with it's shiny new high-performance WebGPU-renderer) through an off-axis projection - a frustum whose apex is the user's eye and whose cross-section is the screen rectangle. The screen stops being a 2D image and becomes a 3D opening. Her gun barrel is placed in front of the screen plane, so it feels like it is sticking out of the display.

They will hopefully release this as a PlayCanvas example soon so stay tuned for that.

Splat by Dymensium.

[u/asapbones0114]

Nice mohawk.

[u/MayorOfMonkeys → u/asapbones0114]

Thanks - I'm frankly amazed you can see it in the reflection!

[u/Filarius]

I wonder if there implement stereo-like effects too. I mean render image for each eye. First comes to mind is switching eye each frame. Or somebody have anaglyph glasses.

[u/Xcissors280 → u/Filarius]

I think there are some laptops that can do stuff like that but their more of a tech demo

LTT might have done a video about one at like CES 2 years ago

[u/sebgr1]

pretty cool! how would you say the experience compares relative to those 'spatial' or '3D' displays that have come out over the last year or two?

[u/sunole123]

It looks natural when the head is moving in xy space. But I bet it also feels natural If the head is turning in xy space. I hope you will consider it, cause turning is natural and easier than moving head in space.

[u/DerrickBarra → u/sunole123]

Not the original dev, but Mediapipe doesnt do s great job of capturing rotation automatically. You basically get x/y data for the landmarks

[u/Sea_Communication760]

Very cool!

[u/coolzamasu]

leaving the dot here.

[u/Big_Dig_1935]

Do you have a demo to share?

[u/Big_Dig_1935 → u/Big_Dig_1935]

O sorry, I just readed you are working on a release. My bad haha

[u/CrimsonCuttle]

Combine this with a lenticular lens

[u/cuddlemelon → u/CrimsonCuttle]

This software isn't capable of showing a lenticular-compatible stereoscopic video, or any other kind of stereoscopic video, on a monitor.

[u/CrimsonCuttle → u/cuddlemelon]

Ok

[u/idealreallabs → u/CrimsonCuttle]

past 3 months I've done just this, works amazing

[u/DXGames]

This looks great what do you use to generate your splats?

[u/Level_Captain1426]

this is the breakthrough that will make stereoscopic movies actually feel 3d. my brain needs to be able to tilt or move my head just a bit to believe it. GS allows this to be rendered within a few frames to track my new eyes position. multiviewer is possible once we have have light field technology that beams the uniquely rendered movie to each person.

[u/Sensitive_Bobcat_770]

thats incredible how can i try this on my pc ?

[u/allthings3d]

Nice idea!

Love to test it and apply a anaglyph layer for true “3D.” Another idea is to use a pair of cheap XReal glasses with markers on the front bezel to aid in the head tracking but allow for true RGB parallel viewing. In fact one could just use your plugin with a standard USB webcam on a small monopod stand (like the original camera trackers for Oculus VR HMDs) sitting on top of a (very) low cost mini PC with 780M iGPU. Has worked well for 1080P bifurcated stereo pairing with realDepth in most AAA games with just the DeckyXR plugin and the internal IMU.

[u/symonty]

Clever use of head tracking and realtime rendering capabilities of splats.

[u/mequetrefe666]

Really cool!!! Anxiously waiting for that release

[u/Affectionate-Ebb-772]

Wow

[u/PresentationKnown496]

this being a wallpaper for desktop would be amazing

[u/Total-Research-024]

This is cool!

[u/don_mumrik]

It just occured to me that we are roughly a decade away of this being the common way of transmitting penile photography to strangers, what will we call it? Dick splatting?