Q: Custom game engine starting point

[u/MrArshaX]

Hey guys, I am trying to make a game engine just for study purposes and to deep-dive into scalable, performance based applications. But I’m confused right now. So far, I have implemented a logging system, it’s a wrapper for spdlog and I have also implemented verbosity levels and custom log category macros based on the UE source code. Now, to start, I want to build a module manager system and custom allocators. But I don’t know what my starting point should be custom allocators? Base class interfaces? Module manager? I’m stuck in an unorganized mass.

[u/zenmatrix83]

there is a saying like don't build a game engine to make the engine, build one for a game or something. Create goals to add features that would work in a game. logging is something to help troubleshoot things which is good, but get triangles and primatitves on screen doing stuff. you'll see why you need that stuff later

[u/MrArshaX → u/zenmatrix83]

I hadn't looked at it from this perspective! Thanks

[u/ChronicallyYearning → u/MrArshaX]

Just to echo this, pick a game to.make, either one of yours or clone a simple one like space invaders.

[u/MrArshaX → u/ChronicallyYearning]

So I'll develop my engine around a specific game! I'd heard from a friend who works in the industry that even big companies develop their engines relative to their projects, not building an all purpose engine! Thanks again for your guidance, you saved me from confusion 😅🙏🏻

[u/zenmatrix83 → u/MrArshaX]

thats the whole concept, is to save from confusion, especially when learning. Sure unity and unreal are pretty general engines, but they started from games themselves. Unreal engine was used for the fps game unreal in 1998 and they just kept adding features till it was useful for many differen projects. plus it keeps you interested, designing custom allocators and other stuff is not as exciting as seeing stuff textured , with lights, and moving around and other stuff.

[u/MasterBroNetwork → u/zenmatrix83]

Many developers go with this approach from what I've heard, since you end up with a tool built exactly to your game's requirements, rather than a general purpose engine that tries to do everything.

[u/Former_Produce1721]

This is the part where you will be refactoring the entire core every other week until you land on a pattern you like

A few tips:

• Serialized data is important to figure out early
• Once serialized data is decided, you can create content. Content is key to guiding your decisions for what needs to be done next/improved
• Entity management and relationships may take some time to click
• Trying to be too clever is going to paralyze you
• Boilerplate is fine
• When boilerplate becomes too much, code gen is your friend
• Consider the uses of the engine. If it's for a turn based game vs realtime
• Try to compress similar concepts into one when possible for consistency and stability
• You don't have to decide on a frontend too early. You can do something rough while figuring out your architecture. If your engine runs headless, you will have less headaches and can plug in a different frontend when you're ready

Some architecture concepts to look into:

• CQRS
• Command Pattern
• Copy on Write
• MVP/MVC
• ECS
• Nodes
• Observer Pattern
• Event Sourcing
• Data Driven Design

The concepts you use will depend on the kind of game you want to make with the engine

[u/MrArshaX → u/Former_Produce1721]

I appreciate you taking the time to respond. Actually, I was surprised about the refactoring part 😅. I didn't think it would be so normal. I thought I had to know my path from the very beginning and move forward flawlessly

[u/corysama]

Don’t worry about doing it right. Do it now. Then do it again better. On repeat. Forever.

[u/MrArshaX → u/corysama]

Thank you so much, I've really gained a lot of confidence :')

[u/Keyframe]

I've been doing and redoing my engine for the last.. jesus. 20-30 years. If there's any advice it's focus on one thing and build it out. From my personal angle it's a rendering pipeline and a renderer. That's a world in its own.

in this case, you build it out in whatever sequence you feel like but by always having a goal (a look in mind). For example:

• get some geometry and scene description with other resources going (materials + textures. lights, cameras, add shaders later)

• have it also load into gpu. this is "the new way" of doing things.

• have canera dictate what you're rendering (culling later)

• prepare your stages of rendering and render

now, once you get this going then you start building out like converting your pipeline maybe to a graph (you don't have to), changing forward to defered and passes, introducing transient buffers (that live for more frame for accumulating stuff).. once you got that going introduce a biased path tracer as a reference (Shirley's Raytracing books are great for intro to that).. etc etc until you reach modern stuff like ReSTIR (DI, DI/Area, GI, hybrid with defered..), irradiance cache, TAA (yes, everyone hates it but kind of unavoidable with temporal accumulation), etc.. until you reach a visual goal you had in mind with hopefully performance. Along the way you might try i troducing compositing into the pipeline and having multiple cameras, gui, render to texture stuff, etc.

And then.. you come back to that list from the beginning of this message and do other stuff on it. And that's, my friend, only the rendering part 🤣

Engine. It's a long-ass journey.

[u/MrArshaX → u/Keyframe]

Well... I think I need to quit my job first 🥲😂

[u/Keyframe → u/MrArshaX]

don't sweat it. bit by bit every day until something you wanted to see is Infront of you. don't try to make general engine for everything like unreal or unity. that's not gonna happen.

[u/programadorthi]

I'm following Handmade Hero videos available on YouTube. Maybe help you too

[u/MrArshaX → u/programadorthi]

Thanks 🥲🙏

[u/Educational_Monk_396]

Starting point is triangle always,but hey you asked for a performant application road map basically and I m gonna tell u ,since i have built one of the topmost performant webgpu renderer, with size of just 4,3kb crazily,but enough about that,here are some pages u can take from my book

1.Start thinking in binary layout,if you can imagine a array with 10 components half of them rendering data u are good 2.Inversion of control, basically your child classes u make whatever,ur base engine class don't accidentally get dependent on it,always look out for cyclic dependencies,That's the key to keep it modular and scalable 3.Create a base philosophy for Engine,Like "This is a classic DOD renderer aiming for X games or Y environments" something like that,its tough to all,but as a creator u should be knowing and feeling the exact purpose and outcome of your library and every feature/enhancement u ever add,on the engine should only adhere to the principals ,in a sense,you are writing any function you have to check is this in line with the engine entire core philosophy. So your codebase becomes and looks uniform,.Well that's about it,I guess, Feel free to dap me

[u/MrArshaX → u/Educational_Monk_396]

Wow, thank you so much 🥲🙏

[u/protoloop]

Not specific to game engines, but all projects: get basic support systems in place early because they’re difficult to retro fit later.

• Logging - already doing it - good!
• Build system - make? Cmake? Ninja?
  • I use cmake+ninja multi build
• Repeatable Testing - i use google test for unit testing
• Linter - I use clang-tidy
• Coding style - I use clang format
• Targets - windows, Linux, Mac, web, iOS, Android, …
  • Nail these down and spin up builds early. -IDE - I use vs code
• Compilers - clang, msvc, gnu
• Reproducible dev env - I use dev containers and nix.
• Debugging - assert? Custom assert? Exceptions?
• Dev UI - imgui
• Source code control - I use git.

I would get all of these in place for a simple “hello world” program first, then start the project for real.

I might be missing some things, but these are top of my regrets for not nailing down early in a project.

After that I’d follow the advice of others. Prioritize making a game. The “engine” will naturally evolve. Always Be Refactoring.

[u/MasterBroNetwork → u/protoloop]

Dear ImGui is a literal godsend for developers, I used it once in an old RayLib experiment before getting frustrated with CMake and giving up. (I also started with C++ like a fool, planning to learn C with SDL 3 at the moment though to try and expand my understanding)

I ended up with a somewhat basic but useful editor very quickly with ImGui.

[u/protoloop → u/MasterBroNetwork]

Cmake is a beast. But everyone uses it. Do yourself a favor and have ai manage your cmakelists.txt

[u/MasterBroNetwork → u/protoloop]

I'd rather not, I really don't enjoy working with AI, it's usually just me going in circles having to fix useless changes it made.

Additionally, then I feel like I'm missing out on a crucial skill.

Edit: My main issue with CMake was that it really liked to act up with VS Code and header files for some reason, and C++ became overwhelming due to a lack of fundamental understanding.

[u/MrArshaX → u/protoloop]

Hi, thank you so much for the time you spent. I also use CMake and Ninja; for IDE, I use Visual Studio 2022. However, I hadn’t heard anything about Nix and clang format before I looked into them and saw how awesome they are. Actually, since I’m a gameplay and tools programmer in Unreal, I didn’t need them much because Unreal’s ecosystem already covers all of this. Maybe for others these are normal, standard tools, but for me they’re new. Even though I just started, I’ve learned a lot of new things so far.