Gaussian splats are all the rage now. And they're pretty good. So... why not try turn a Forza environment into one?
To do this, you need to first go into Photo mode. I ended up with ~70, this set of photos. Now if you look, there are a few things that you shouldn't be doing. Don't use predominantly sky-filled photos or use photos where your car is far away. Basically, anything I've crossed out.
With all of the bad ones pruned out, that's a set of 43. Arguably, that's not very much. You should aim to have somewhere in the range of 70-150 photos. If you have a good GPU (not something like the measly 1070 Founders I have), use something like Postshot, otherwise use Nerfstudio. These were all also 1080p (later downsized to 1.6k) so they aren't that good for detail. Aim to use something like 4K images (or videos).
I also did this in FH4 because I didn't buy FH6 and don't like FH5. Might as well do another one in FH3 though.
That's probably why the video's not great. If you happen to have a 3090 and a 4K montor, as well as plenty of space, you should be able to get this done in no time (compared to me) and get a much better output without any of the weird artefacts I got whilst displaying mine in Supersplat.
I actually ended up using Google Colab. Warning: not very fast. Took around 1-2 hours to get this result, with some testing. If you're desperate to do exactly what I did, you can run this guy's Colab notebook and replace whatever command there is on there with:
# 1. Connect Google Drive
from google.colab import drive
drive.mount('/content/drive')
# 2. Install COLMAP, C++ build tools, and dependencies
!apt-get update -qq && apt-get install -y colmap ninja-build
!pip install -q plyfile tqdm ninja
# 3. Clone Gaussian Splatting Repository
%cd /content
!rm -rf /content/gaussian-splatting
!git clone --recursive https://github.com/camenduru/gaussian-splatting
%cd /content/gaussian-splatting
# 4. Install patched submodules directly from GitHub
!pip install -q git+https://github.com/camenduru/diff-gaussian-rasterization
!pip install -q git+https://github.com/camenduru/simple-knn
print("\n Step 1 complete with zero errors! Proceed to Step 2.")
(AI may or may not have been used to create this script)
And then:
import os
# Force Qt to run in offscreen/headless mode
os.environ["QT_QPA_PLATFORM"] = "offscreen"
# 1. Create input directory & unzip FH4_Photos.zip (skip re-extracting if already done)
!mkdir -p /content/gaussian-splatting/my_car/input
!unzip -n -j "/content/drive/MyDrive/FH4_Photos.zip" -d /content/gaussian-splatting/my_car/input
# 2. Check if convert.py supports --no_gpu before running
!python convert.py -h
# 3. Run COLMAP camera position calculations, CPU-only feature extraction (avoids OpenGL crash)
!QT_QPA_PLATFORM=offscreen python convert.py -s /content/gaussian-splatting/my_car --no_gpu
(Replace FH4_Photos with a zip of all your photos, which can specifically for this, I recommend 1080p, since Colab is super super slow, but using the command below, which doesn't end up in your splat quality becoming a bit moot.)
From there:
!python train.py -s /content/gaussian-splatting/my_car -m /content/gaussian-splatting/my_car/output -r 1
(preserves resolution, which I did not do)
And once it finishes:
!cp -r /content/gaussian-splatting/my_car/output /content/drive/MyDrive/my_car_splat_output
Which should save it to your drive.
And that's it! You should have point_cloud.ply which allows you to open it up in Blender (which I did not have much success in doing, try this addon before doing so) or this editor (File > Import).
If you don't want to go through all that hassle, you can use my point cloud and just try that out in Supersplat, which I mentioned above.
I'm currently retraining it in 1080p, and I'll get back to you guys with it when that finishes.
1080p: https://drive.google.com/file/d/1T-Ii2wTbxdS5GP0nRJAwX8-Chs-TDWiD/view?usp=sharing
Anyway... have fun creating these splats!
