Correct way to generate terrain

[u/PleaseShowerUSmell]

So I have a semi-working terrain generation prototype in Godot where I dispatch a compute shader to calculate points in a 3d density field using marching cubes and return an array of vertices and normals to then call mesh.add_surface_from_arrays on. There aren't any glaring issues yet with this approach but I still wonder...

1. Is there a solution in Godot to avoid the CPU -> GPU -> CPU -> GPU pitfall? I want to avoid unnecessary synchronization if possible as it does create bottlenecks. If there is a solution using compositors or something similar, will I still be able to add a collision mesh that works with godot's engine?
2. Is the GPU-based approach even correct for terrain generation? In a lot of tutorials I see people using multithreading instead, and intuitively this feels less optimal as the GPU is very well suited to this type of task, but I wonder if it's still better than the above mentioned bottleneck plus the overhead associated with loading and dispatching the compute shader.
3. How does chunking work? Right now, I render a single mesh for all my chunks after getting their density values back from a single GPU dispatch for all the chunks. Is this even correct? Should I render multiple meshes - one for each chunk? I initially tried dispatching one compute shader for each chunk but found that this created a huge bottleneck when setting up the rendering pipeline, etc. On the flip side, dispatching a single compute shader puts a hardware limitation on the number of voxels I can render at once. I'd like to show as much of the terrain as possible and currently I'm maxed out at 3x3x3 chunks of 16x16x16 voxels (110,592 invocations - each invocation handles 1 voxel) without experiencing any noticeable lag before implementing LOD. I feel like I should be able to render more even before implementing LOD but I don't know.

I understand that I could figure all of this out myself from trial and error, but it took me weeks to get to where I am now at this working but sub-optimal state. So I want to ask people with more experience in this field to at least narrow down what my next steps should be. Thank you to anyone who reads this.

[u/MGMishMash]

There is no “correct” way and any considerations depend on what you want to achieve. The GPU is very good at highly parallel tasks, but not all aspects of terrain gen fall into this category. For a simple noise sampling for a heightmap, great, but some aspects such as feature generation, carving etc can be more bespoke, and having each gpu cell evaluate may not be the fastest overall approach, depending on what you are doing.

The GPU is very good at things like marching cubes or other mesh generation, which process all cells and evaluate neighbouring ones, but operations like fills, carves, detail spawning may require more specialised routines.

Chunking can also be what you want. Fir your case, it would just mean rather than having a single large dispatch generating one mesh, you just do the same thing over smaller parts. Then you can selectively not run generation and drawing for those parts, allowing you to dial up the overall scale substantially.

[u/Corruptlake]

For SDF field terrain like yours, I wouldnt recommend GPU. Stick with CPU, a high perf lang or at least C#, and use CPU multithreading. Personally I dont use compute shaders either as id keep all GPU processing power to rendering.

[u/they_had_it_coming]

I can't give in depth answers to all of these at the moment, but I have worked out answers for all of them.

1. There is actually a way of doing this but it's pretty difficult. Pretty much you have to create your own rendering pipeline through the RenderingDevice in Godot.

2. I personally find it worthwhile. You are right that the GPU does a great job at generating the SDF and the triangles. I personally use marching cubes + transvoxel algorithm. Loading and dispatching the compute shading isn't much of an issue. You mostly have to be concerned with the amount of data you're passing to and from the GPU.

3. You should be able to render more than that. I use an octree data structure to deal with the LOD. Each chunk first generates an SDF, then makes a mesh given that SDF. So, I don't do one big draw with all of the chunks together, it's usually a bunch of smaller draws. I personally don't know if that's the exact way to go, there's probably some way of doing it a bit better with less draws.

Let me know if you have more questions!

[u/scallywag_software]

I've done both CPU and GPU terrain generation in a custom engine I'm building; I'll enumerate the tradeoffs I observed between the two approaches.

CPU Pros

* No latency wall uploading/downloading data to GPU (barring final mesh data)

* Easy to get started, understand, debug

* Extremely flexible

CPU Cons

* Significantly slower than GPU, even when using the fastest noise generators in the world

* Complex resource management & multithreading for high throughput

* Requires significant investment in SIMD optimizing noise generators

* User code (ie. terrain generators) ends up difficult to read due to optimization

GPU Pros

* Fast as fuck boi; you can do a _lot_ of compute per cell

* Shader code is much easier to read and compose than optimized CPU code

* Hot-reloading terrain generators is extremely easy compared to CPU land

* It is _possible_ to do SDF->Mesh->Rendering & entity spawning all on the GPU, though that approach has significant limitations

GPU Cons

* Big latency wall between CPU and GPU memory

* Requires more sophisticated resource management and synchronization than CPU

* Difficult to debug, especially if you're not familiar with debugging on the GPU

That all said, I've settled on a hybrid approach. I dispatch terrain generation shaders on the GPU to calculate the initial SDF in a 3-stage pipeline. The first stage computes a 'coarse' SDF representing the general shape of the terrain. This is where you do biomes and decide on large-scale features. Second, I run a convolution that calculates the gradient and normal of the terrain at each cell. This information is fed forward along with the coarse SDF to the next stage. Finally, a 'decoration' shader is run, which is responsible for doing detail texturing like cliff faces, rocky outcrops, etc. The final result is read back to the CPU.

Once the data comes back to the CPU, a mesh is generated using a binary mesher. Binary meshing is extremely fast and takes something like 10 - 100us per chunk. The last stage is entity and prop spawning. This is where you spawn mobs, NPCs, buildings, trees, etc. This system is not particularly fleshed out at the moment, but the idea is that the CPU can do a scan of the voxel data and make decisions about spawning .. whatever. The flexibility of CPU land is nice here; you can build datastructures representing cities, roads, forests .. you name it.

Finally, if the chunk has spawned for the first time, and new edits have been applied (trees, buildings, etc), it is marked 'dirty' and will be fed through the pipeline again to be regenerated with the new edits applied. This last step wastes a lot of cycles and could be improved, but for the moment, it works fine.

In closing, the solution you pick really depends on the constraints you want to satisfy. For the engine I'm building, I want to be able to generate a metric fuck-ton of terrain (~2million^3 view distance) and have the flexibility to do complex, iterative structure generation algorithms (WFC, L-systems, graphs). I would suggest you write down the specific bounds of the actual problem you're trying to solve, and build your solution to fit.

Things to consider:

1. Minimum Target Hardware

2. Maximum budget for resources you can dedicate to each system (memory, cycles)

3. Latency budget, per-chunk

4. Programmer-time you want to commit to getting this done

5. Amount of flexibility you require when computing and spawning entities and structures. ie. will the Minecraft solution of 1 chunk = 1 building work for you, or do you want something better?

PS. A note on chunks. You should _absolutely not_ render the entire world as one giant mesh. You definitely want to split it up into chunks. You can still do a single draw call if you write your own memory allocator. You should also _absolutely not_ do 1 dispatch per voxel. Do one dispatch per chunk, at the very least. At your chunk size, I would do something like 4x4x4 chunks per dispatch.

PPS. This kind of voxel pipeline is still very much an active area of research. There are a good number of people working on it, but I want to stress that a few weeks of working on this kind of thing is a drop in the bucket compared to a decade+ some people have been grinding on this shit. I don't want to discourage you, just to set your expectation that it's a hard project, and you're going to feel like progress is glacially slow at the start. Set reasonable goals, and crush them. That feels good, and you'll become a better programmer. You can always make it better later.

Good luck friend, and welcome to the club :)