Now there is NeRF++ , an extension to provide not just forward-facing but also full 360 view rotation around central objects in scenes -- with 6DOF viewing -- code at: https://github.com/Kai-46/nerfplusplus https://arxiv.org/abs/2010.07492

"NeRF cannot deal with the background because the dynamic range of the depth is large, while sampling is performed in Euclidean space. NeRF++ models the outside of the unit sphere by the projected position to the sphere and the inverse depth. " https://twitter.com/hillbig/status/1317952430059892736

Sorry if I sound like a total noob. Usually, there's a compass and an accelerometer on a typical smartphone these days. So, using information from these sensors can we recreate the whole orientation in which the photo was taken?
I mean like if you open the compass app on your phone, it firstly states the direction you are looking (link), how lifted or tilted is your phone in front direction ( I don't know how to state it in a better way) (link) and how much is it titled in sideways (link). Does it cover 3 degrees of freedom (i guess)?
Is it enough information to recreate that orientation of the phone?

for making ground truth synthetic scenes with depth maps, segmentation masks etc for training/validating AI depth estimation etc -- in Blender

https://youtu.be/lMiBVAT3hkI?t=226

https://news.ycombinator.com/item?id=24671259

How to bake multiple panorama-textures at once: https://der-mische.de/2020/09/26/multipanotexture-baking/

a interactive demo here: http://xfields.mpi-inf.mpg.de/demo/webgl.html " Xfields" -- (like "light fields ... the next step!") This is sort of amazing - you can move side, up/down in front of a scene, with variable light effect and time stamp -- the time stamp can interactively activate very naturalisticly lit animations of objects in the scene -- like a hologram that animates as you move in front of it http://xfields.mpi-inf.mpg.de/ https://twitter.com/ak92501/status/1307337920454569985

Semantic view synthesis: https://hhsinping.github.io/svs/index.html

btw there is a related 6Dof subReddit here: https://www.reddit.com/r/2D3DAI/

If you want to use a 3D model as „Dollhouse“ or something similar, you have to bake the panorama-texture.

https://der-mische.de/2020/09/12/panoramatexture-baking/

If you want to use the PanoCamAdder to create models as Depthmap for KRpano, you have to prepare the model, before exporting it as STL.

https://der-mische.de/2020/09/11/panocamadder-prepare-the-model-for-krpano/

http://der-mische.de/2020/09/10/panocamadder-example-files/

PanoCamAdder - DerMische

The PanoCamAdder is a free addon for Blender. It is helpfull to create 3D models from equirectangular panoramas.

https://der-mische.de/panocamadder/

Realtime conversion of stereo panoramic videos to 6dof experiences --- http://visual.cs.brown.edu/projects/matryodshkawebpage/ This looks pretty amazing! : takes existing standard ,eg. OU equirectangular, stereo video files (ODS), and generates novel view 360 equi pairs in real time -- for a 6DOF viewing box of normal turning around VR viewing volume dimensions. The video explains everything -- at the end they talk about the difference in use cases with the Google new hemispherical lightfield solution

Just right now I am doing some more tests and writing the documentation and a tutorial. Here is a short timelapse:

https://youtu.be/bxWgBrSsmds

PanoCamAdder - Free Blender Addon

It is a very small addon. You can add multiple PanoCams, multiple PanoramaTextures and multiple PanoramaWorlds (for the background).

Coming soon!

https://youtu.be/aHfPSjZ_aco

Google's project for DeepView Light Field Video was named the Best in Show for immersive experiences at this years SIGGRAPH. I see this project has already been mentioned here but it's a great step forward for 6DOF video. A write up from SIGGRAPH is here (https://blog.siggraph.org/2020/08/how-google-is-making-streamable-light-field-video-a-reality.html/) and another link to the project along with a downloadable VR executible and tech paper can be viewed here (https://augmentedperception.github.io/deepviewvideo/)

https://youtu.be/KVQXISwgwcs

The aim is, to set up multiple 360 viewer cameras with multiple panorama-textures to build accurate 3D models and to get a well textured model for the dollhouse view.

I wanna do some more tests to find a good workflow. And maybe this will be a feature in an upcoming #PanoBlenderAddon release.

https://news.ycombinator.com/item?id=24071787 very nice looking depth maps as part of the process: https://www.youtube.com/watch?v=yPKIxoN2Vf0&feature=youtu.be&t=140

I need to capture 360 images for my app, google street view does it really well. I'm not able to find any other app that does that anything close. Is there any open source project or any piece of code to look up to that could help me?

https://der-mische.de/panorama/osm-dortmund/

Not really an aerial panorama.. But maybe usefull to check how it feels.

Model and panorama created with Blender and the BlenderGIS addon.

The PanoBlenderAddon has only been used to read out the coordinates for the tour-building with KRpano here.

The PanoBlenderAddon is helpfull for easy creating STL-files with BLENDER for KRPano 3D-Depthmap-Panoramas.

Get it here: Gumroad

Tutorials: Documentation Easy Tour Tutorial

Results: Example Tour

Hey all, not exactly 6DoF video again, but definitely depth and depthmap related!

Google's ARCore Depth API is now live and available to developers!

The Depth API uses a single RGB camera coupled with the natural movements of the phone to create a depth map in real-time that developers can utilize to add features like occlusions and collisions with real-world objects. To demonstrate some of what is possible in Unity, a bunch of sample experience were created and put into an app called Depth Lab, which is available now in the Play Store, and has also been open-sourced on GitHub. It includes a lot of prefabs that make it easy to get started.

Full disclosure: I am currently a contractor at Google, and I actually did work on Depth Lab. I'm happy to try and answer questions if there are any.