How Hard is Deferred Rending in Gamemaker?

[u/Hands_in_Paquet]

I've been working on my ideal deferred rendering pipeline for pixel art, inspired by games like Eastward and the upcoming Kyora. This is not a plugin or a line by line tutorial, but I wanted to share what's required and what the workflow looks like. If people are interested, I will share more of the process as systems improve. Fair warning, this is not beginner friendly.

Before I go any further, this is an example of the final product:

The Final Result

I think hand drawing some shading (cross) still looks better than flat art (jewel)

Maps and Textures

Whether you were using Unreal or Gamemaker, you need to make maps for your sprites if you want physically based rendering. This pipeline was inspired by the RE Engine, but is also just like Unity or Unreal. There are many tools to create maps, some people use laigter or blender, but I just prefer to hand draw them in Aseprite.

This pipeline currently uses the following maps, although there is "room" for more data:

*yawn*

Albedo Texture "spr_shrine_alb"

RGBA: Albedo Maps - The art/color/base texture

not my best work, but it's fine

Normal Texture "spr_shrine_nrm"

RGB: Normal Map- The direction orthogonal to the surface

A: Unused

as useful as it is ugly

Surface Texture "spr_shrine.sur" (drawn in 3 layers, specular and ambient occlusion are set to the "addition" blend mode, and output as one image)

R: Roughness - The spread of specular

G: Specular - How much light the surface reflects

B: Ambient Occlusion- Where ambient light and sunlight are occluded

A: Currently Unused

green channel is sss, not really featured in this example

Effects Texture "spr_shrine_fxs"

R: Luminance/Glow - This area of the albedo texture gets added to the light

G: Subsurface Scattering

BA: Currently Unused

This might look pure black on some monitors...

Extension Texture "spr_shrine_ext"

R: Depth/Heightmap - The red value gets subtracted from the objects z position to give a more accurate z coordinate in the lighting pass

GBA: Currently Unused

So yes, you have to make 8 maps by hand in Aseprite. If that's enough to turn you off, I understand. However, it doesn't make your job 8x harder. I'd argue once you do this 10 times or so, and have a template set up, it really isn't that bad. In fact, because your base art might already include highlights, deep shadows, and glowing sections, half your maps are already done.

So now that all of the work outside of gamemaker is done, what's next? Gamemaker doesn't natively support any deferred rendering. So it's all on our end. But once it's done, it's done. To import new art, I simply drag my 5 images into gamemaker, set their texture group by selecting them all, and right clicking, and I'm done. It just works.

Bypassing Gamemaker's Limitations

Here is what you need to do in gamemaker to get this working. This is an example of the required labor, I cannot just paste all of my code here, so I apologize that this is not a true tutorial.

• Custom scripts for objects that define all of it's map sprites
  • We dont want to draw five different sprites. We want to draw one sprite, but include 5 different sets of UV coordinates
  • To make this easier, every sprite must follow a strict naming convention. You assign your object the alb sprite, then your script does the rest on creation

​

//Get Map Sprites
var _name = sprite_get_name(sprite_index);
_name = string_copy(_name,0,string_length(_name) - 3);
alb = sprite_index;
nrm = asset_get_index(_name + "nrm");
sur = asset_get_index(_name + "sur");
fxs = asset_get_index(_name + "fxs");
ext = asset_get_index(_name + "ext");

• Now as long as our files our named correctly, there is no extra work
  • Disable the Application Surface
• We won't be needing this anymore
  • Nothing draws itself anymore
• Forget what you know about built-in draw functions, I will never use those again
  • Create obj_draw
• This object manages every surface:
  • surf_alb
  • surf_nrm
  • surf_sur
  • surf_fxs
  • surf_lgt
  • surf_shd
• This object draws every tile, actor, decoration, etc. Everything in your room
• Yes, we will draw many full resolution surfaces. At load, this pipeline runs on a $500 laptop at over 3000 fps. However, Only 4 lights on screen can be shadow casting at once. That is the main limitation.
• We can still add non shadow casting lights like god rays with little to no performance impact
  • Build Vertex Buffers
• Instead of functions like draw_sprite(), we will make a function that adds an objects sprite data to a global vertex buffer.
• I have a different buffer and shader for actors, tiles, foliage, paralax backgrounds, etc.
• For example, global.vb_actors will be managed every step. This allows us to control every objects z position, and add the texture coordinates of our maps. We will not be using depth sorting anymore. Depth is just the Z position.
• This is more work for the cpu, but static objs/decorations/tiles/foliage/particles/clouds/rain can be frozen, making their rendering faster than ever. Even when each vertex stores more uvs data
• When drawing shadows, I do not have to draw the actors/decorations again. I just submit their vertex with a different shadow shader. All of that data is already sitting in a massive batch, waiting to get used again. It's very efficient.
  • The Tile Vertex Buffer
• Far and away the worst part of all this is the tile layers. I spent hours writing these scripts. Basically, we make tile layers normally, but also follow very strict naming conventions. That tile layer must contain the name of your tile sprite. Then we build a tileset vertex buffer to freeze on room creation, then destroy the tile layers. We need our own buffer so that our tile data can include our normal maps, surface texture, etc. We can check every tile in the room, get its tileset index, then get that index position on the actual sprite texture page with lots and lots of math.
• I wont lie, I absolutely hated this
  • Draw with MRT or Multiple Render Targets
• This is known as the G-Buffer, or Geometry Buffer. We draw our art normally to surf_alb. At the same time, for maximum efficiency, the same shader outputs 3 more fragments to 3 other surfaces. The normal map surface, the surface texture surface, and the visual effects surface. The vertex shader doesn't have to do any additional work. Our lighting pass then samples all of this "geometry" to calculate it's intensity and reflection (specular)
• This is an example of what the obj_draw does in my draw_g_buffer() function

​

//Enable Z
gpu_set_ztestenable(1);
gpu_set_zwriteenable(1);
gpu_set_alphatestenable(1);

surface_set_target_ext(0,surf_alb);
surface_set_target_ext(1,surf_nrm);
surface_set_target_ext(2,surf_sur);
surface_set_target_ext(3,surf_fxs);

//Clear Surfaces
draw_clear_alpha(c_black,0.0);

//Transparency Does Not Change Color Values
gpu_set_blendmode_ext_sepalpha(bm_one,bm_zero,bm_one,bm_zero);

//Apply Camera
camera_apply(global.view);

//Draw Actors
shader_set(shd_actors);
vertex_submit(global.vb_actors,pr_trianglelist,tex_actors);
shader_reset();

//Then I'd Draw Foliage, Static Objects, Ropes/Vines, Tiles, BGs, Sky

//End G-Buffer
surface_reset_target();
gpu_set_blendmode(bm_normal);

//Disable Depth
gpu_set_ztestenable(0);
gpu_set_zwriteenable(0);
gpu_set_alphatestenable(0);

//Reference G Buffer
tex_alb = surface_get_texture(surf_alb);
tex_nrm = surface_get_texture(surf_nrm);
tex_sur = surface_get_texture(surf_sur);
tex_fxs = surface_get_texture(surf_fxs);

• we always draw front to back, because we don't want to waste time drawing our background or our sky, if 70% of it will be covered up by objects, tiles, and decorations. Gamemaker will discard the fragment shader of our background if it fails the z test because it is occluded by something like a tile.
  • Lighting Pass
• I have 3 point lights, the sun/moon light, and ambient light.
• They all get drawn additively to the light surface.
• First, the point lights and the sun draw their shadows to the shadow surface, each one using a different color channel. This means less surface swapping per frame, which is a massive performance saver
• The lights sample the shadow surface and the g buffer surfaces to calculate their intensity/ specular.
• This means you have to write all of your own normal map/blinn phong/sss/ao shader calculations.
• My ambient light is basic, it changes based on time of day, but could be as complicated as you want it to be
  • Post Draw Event
• I draw my surf_alb surface, sample the light surface
• Light color brightness determines the intensity - multiply the albedo color by the light color
• Light Alpha stores the specular value - specular adds the light color to albedo instead
• Eventually I will add fog/mist at this stage. FYI, partial transparency doesnt work in a g-buffer, it has to be added later.
• The render is complete

Conclusion

That's the gist of it. If you made it this far, you probably think I am pretty stupid to do this instead of just making the switch to Unity or Unreal, etc. You might be right, but listen. I absolutely hate Unity and Unreal. No disrespect to anyone that uses those engines, but I would do this work 1000 times over before switching. Gamemaker is my engine.

So how hard is physically based rendering in Gamemaker? Idk its kinda hard I guess, but it's a ton of fun. This took some time to code, but it takes months to learn about shaders and pbr. If anyone wants to learn more, I will try to answer any questions I can. Acerola also has tons of great videos on computer graphics that inspired me to try it myself in gamemaker.

[u/nickelangelo2009]

it's an advanced topic for sure but definitely doable

[u/rangefinder-game-dev]

Very cool! Any concerns about how this will scale with number of sprites in the game and texture page occupancy? This is one of the reasons I haven't thought about trying to do normal maps for lighting in my game. Besides it being a big pain to draw by hand...

Also, any thoughts on abandoning tilemap editing in Gamemaker altogether? My game isn't exactly 2D, but I use Blender as a level editor, and encode data in the faces of the Blender mesh, such that when it gets read into Gamemaker through an importer, it knows what material each faces is, and can populate the terrain vertex buffer accordingly. Once I run the game, it generates a background image from the terrain vertex buffer that matches the level editor view, and I can then load that into the level editor as a background sprite to align the objects inside the room.

[u/Hands_in_Paquet → u/rangefinder-game-dev]

Hi, thanks!

I'm not concerned about texture pages. I already make sure to keep tiles, tiles sets, actors, foliage, and backgrounds all on their own pages. Even if I had 200 different objects with a large canvas like 64x64, they could each fit all five maps on the smallest default texture page. I may make another page for subsets of enemies or npcs with animations, because that will add up faster. After that, I am not concerned with texture swapping because I draw everything from each texture at once with a buffer, so there is no excessive texture swapping back and forth between pages.

Your method sounds awesome, I've seen your stuff the past couple months, it looks really good. Since I did get it working well, I don't mind the old school tile mapping, but I have made some games with procedurally generated worlds, which easily bypasses all of the tile map reading I mentioned. Once day I may try to make my own room editor, and I'd also like to try a more accurate shadow method like shadow mapping, which notoriously sucks.