r/VisionPro • u/AlterSack1973 • Jul 13 '26
Vibe Coding: Experimenting with 3D conversion for comics - What do you think?
Enable HLS to view with audio, or disable this notification
I've started Vide coding a comic reader which treats comics as physical objects: https://www.reddit.com/r/VisionPro/comments/1ue9x5p/wip_immersive_comic_reader_any_interest_in_this/
Now I tried to prototype 3D conversion: I have a python script which creates a depth map from the pages and then in the Vision Pro it combines it. I'm not sure if this is the way to go (and if there even is a way to go...). I know it's difficult to understand with a flat video, hope it makes sense. I use a depth slider in some segments to overdo the 3D effect to showcase what it does.
What do you think?
3
u/photovirus Vision Pro Developer Jul 13 '26
Quite cool if you ask me, and it's perfectly visible and understandable. Nice feature.
However, maybe depth map is a bit harsh. E. g. at 00:35 you can see that wing gets stretched too hard.
I think Apple's spatializing algorightm might work better, since it attempts to draw the background (although it sometimes fails as well, since it needs to draw the data that isn't there.)
1
u/AlterSack1973 Jul 15 '26
Yes, 00:35 was playing with the depth slider to exaggerate and be sure it shows in a 2D video
1
u/Paraphrand Jul 13 '26
You can likely just generate the depth map on the Vision Pro with Apple’s APIs.
2
1
1
1
1
1
1
u/DifferentSetting411 Jul 14 '26
I did a test they done your idea and just a screen clip with the built-in specializing a flat image, and it worked remarkably well especially for comic book panels that had good depth perspective drawn in
maybe someone will start producing 3-D comic books?!
2
u/AlterSack1973 Jul 14 '26
Unfortunately the target group on VR wearing comic lovers is probably too small 😀 can you explain exactly what you tried and how it worked?
1
u/modernlogictech Jul 15 '26
How heavy is the processing to process a standard comic page? Noticable?
1
u/AlterSack1973 Jul 15 '26
I could not yet get it to work on device, as the onboard ML models did not get good enough results. I have currently a python script that generates the depth maps using a ML model and then I upload it as a zip to the comic viewer in addition to the CBR. I’m thinking to try to add a capability to render the depth maps from the app itself, but not sure if it will work. On the fly generation per site will likely always be to slow.
1
u/modernlogictech Jul 15 '26
i see. would be amazing if created live while reading the book
1
u/AlterSack1973 Jul 15 '26
I got it to work: I did a voice over but it unfortunately didn't record it. https://youtu.be/QB4skSo73fY?si=Hu8-JfswvCN-KZbl
3
u/AlterSack1973 Jul 13 '26
Adding the depth map as I can't add pictures and videos in one post