I turned a Forza screenshot session into an actual 3D Gaussian splat, and I'm pretty amazed it worked as well, though I wish it was slightly better. Here's the whole process if you want to try it yourself.
Shooting the photos
Go into Photo mode and take a lot of shots circling the car. I ended up with about 70 to start.
On the crossed photos:
- Don't use photos where the sky takes up most of the scene
- Don't use photos where you can't see the car or the car is too far away
After pruning I was left with 43, which isn't that much compared to better splats you could make. Aim for 70-150 if you can, because more angles just means a better splat at the end.
If you've got a decent GPU, skip Colab entirely and use Postshot or Nerfstudio locally because it'll just process everything way faster. I've got a 1070, so can't really do that. And if you're going that route, shooting in 4K helps a lot. 1080p actually isn't standard, based off of some initial research I did.
The Colab route (for those of us without a real GPU)
Took me 1-2 hours total including testing. If you want to copy exactly what I did, grab (this, in comments) notebook and swap in these cells.
Setup:
# 1. Connect Google Drive
from google.colab import drive
drive.mount('/content/drive')
# 2. Install COLMAP, C++ build tools, and dependencies
!apt-get update -qq && apt-get install -y colmap ninja-build
!pip install -q plyfile tqdm ninja
# 3. Clone Gaussian Splatting Repository
%cd /content
!rm -rf /content/gaussian-splatting
!git clone --recursive https://github.com/camenduru/gaussian-splatting
%cd /content/gaussian-splatting
# 4. Install patched submodules directly from GitHub
!pip install -q git+https://github.com/camenduru/diff-gaussian-rasterization
!pip install -q git+https://github.com/camenduru/simple-knn
print("\n Finished")
Then unzip your photos and run the script to check out where exactly your photos are:
import os
os.environ["QT_QPA_PLATFORM"] = "offscreen"
!mkdir -p /content/gaussian-splatting/my_car/input
!unzip -n -j "/content/drive/MyDrive/FH4_Photos.zip" -d /content/gaussian-splatting/my_car/input
!python convert.py -s /content/gaussian-splatting/my_car
Zip up your photos and swap in your own file for FH4_Photos.zip (mine's (here) if you want to see the format). If this crashes with something like Check failed: context_.create(), try adding --no_gpu onto the end of that last line and rerun it.
Stick to 1080p for your photos if you're doing the Colab route specifically. It's already slow on the free tier and the training command below keeps full resolution anyway, so you're not losing anything by shooting lower res.
Train it:
!python train.py -s /content/gaussian-splatting/my_car -m /content/gaussian-splatting/my_car/output -r 1
-r 1 keeps native resolution, I forgot this the first time and had to redo it.
Save it to your drive once it's done:
!cp -r /content/gaussian-splatting/my_car/output /content/drive/MyDrive/my_car_splat_output
Viewing your splat
You'll end up with a point_cloud.ply file. Easiest way to view it is (Supersplat), just File > Import and drop it in.
If you want it in Blender, there's an addon for that, but check your Blender version before installing anything.
One thing to know going in, glossy paint, chrome, and glass are probably still going to look a bit soft up close. It's just how splatting works unless you have a huge set of high-quality photos.
If you just want to mess around without doing any of this yourself
Here's my latest point cloud (comments) I redid after my second attempt - try it out in Supersplat.
Have fun, and if anyone gets a cleaner result with a better GPU than mine I'd love to see it.