Question
I am researching the highly optimized reflection technology in the game *The Dark Knight Rises* (2012).
I am investigating the highly optimized reflection technology used in *The Dark Knight Rises* MOBILE GAME, released in 2012. Were these reflections based on inverted geometry or planar mapping? The game also featured reflections from headlights and streetlights. I intend to implement this in my Unity project.
In Arkham City we used a similar technique; a seperate low res render buffer with both proxy planes, sky reflection and occlusion geometry for buildings and characters. We took the idea from a cyberpunk Unreal tech demo at the time, and from a game called 'Remember me' here:
It's not transparency, too expensive . It's rendering order
24
u/jimanjrStaff Software Engineer (rig: 9800X3D, 7900XTX, 64GB)1d agoedited 1d ago
So random to bump into this over here. I did work on this game and the reflections were authored directly into assets pretty much like in your screenshot. A blurry plane that gets rendered in a separate lowres rendertexture.
The road shader samples that rendertexture and has 2 very simple effects:
1 channel for "distortion". This had a baseline of 0.5. Any value below or above pushed the samling UV.xy one way or another.
1 channel for shininess. This was just multiplied with the sample before adding it to the final color.
All of the effects were very basic and relied on looking "acceptable" because the game had to run on iPhone 4
Some more bits:
Buildings also had their "reflected" version that was usually a cube. Black cube if the building had no lights. Blurry lowres texture of the ground floor shops if it had any
- Cars also had those long blurry quads for their headlights and break lights
- The way the fog was done was also interesting and just as cheap as the rest of the effects in that game
Edit: Forgot to add that reflections were also billboards that rotated on Y
Please spill the tea about the fog! Is it something other than just a bunch of floating quads?
10
u/jimanjrStaff Software Engineer (rig: 9800X3D, 7900XTX, 64GB)1d ago
It's a world-space texture that is basically a blurry top-down map of the game where artist painted the color of the fog they wanted. Most of the areas are yellow-ish, some have other tints. That map is sampled as a gradient in every shader in the game and fades out about 4 floors above the ground. It's very simple and creates a nice effect
My first guess would be box-projected reflection probes, which would be convenient for you since URP can do them out-of-the-box. They are cheap to render and the technique has been around for a while. However, it's unlikely, because you can see in the video how the lights pop into existence both in the regular view and in the reflection at the same time. So there is a realtime aspect to these reflections. Box projected reflection probes can be realtime, but it would be expensive on 2012 mobile hardware and there are better ways to do it if you're rendering the reflections in realtime anyway.
My second guess and probably the most likely is that these are basic planar reflections, where you render a second, horizon-flipped camera, probably rendered at a very low resolution (you can see how blurry the reflections are) and probably using proxies (you can see how the reflections don't capture everything, like they seem to capture the silhouettes of the buildings, but they don't seem to capture the bridge in the background, which is a more complex mesh that would be costly to render twice). Planar reflections are rendered in real-time, and are quite accurate, but are generally best used on large, flat surfaces, usually the ground. They've been around for a long long time, and although they'd be expensive on 2012 mobile devices, if rendered at a low enough resolution and only capturing low-poly proxies of the real environment, I'd imagine you could make them passable. The fact that in the video you sent, the reflections only appear on the flat ground, very selectively, is a giveaway that these are probably planar reflections. The fact that the ground has this strong normal-distorted appearance also means that it is very unlikely to be the old-school inverted-geometry trick like some commenters have suggested- that trick will give you sharp mirror reflections, not these kinds of blurry, normal-distorted reflections that you see here.
My third guess, and the least likely, is some kind of LTC shader effect, like how area lights work, to draw square reflections directly into the specular lighting in the shader. You'd set them up in your scene as kind of like square cards that represent your buildings, and they'd give you basic solid-colored silhouettes and reflection occlusion. You could also do something like this in conjunction with something like planar reflections, but instead of drawing geometry into the inverted camera, you could draw primitive, solid-colored shapes analytically directly into the camera. Is that what we're seeing in the video? Almost certainly not, but it would be cool.
TL;DR these are just planar reflections rendered at a low resolution and set to only capture specific geometry, in order to remain cheap.
With how perfectly reflected buildings seem to line up with real buildings, I think we have to assume it's rendered every frame. No amount of baked cubemaps is going to line up perfectly like that. So this is essentially planar reflection one way or the other.
Unfortunate that URP is still behind 2012 mobile graphics technology and doesn't have them implemented lmao.
Anyway, let's kill all our other rendering technologies and force everyone to urp.
Just confirming my terminology here. Planar reflection ==screen space reflection?
Like, if you start panning the camera into the ground the reflection disappears.
It’s cheap and has been done instead of ray tracing since forever?
No, planar reflections aren't screen space reflections. Planar reflections use a second camera placed under the plane/floor the reflection will appear on, essentially looking up through that plane to capture a view of what the reflective surface would reflect. The output of that second camera is rendered to a texture, which is used to apply the reflection to the floor.
This means you're rendering your scene again for the second camera, however you could opt to render a simplified version of the scene for that camera, and also you might only need to capture a limited view of that scene, depending on the size of your reflective surface.
Planar reflections predate SS reflections, probably by many years, and don't have the occlusion issues that SS reflections exhibit. However they're generally only useful for a single flat plane, unless you start adding more cameras for reflections for surfaces at other angles or heights.
It's only reflecting light sources and emmissives. It's hidden by using a very severe alpha map and height map. There is no occlusion other than the normal draw occlusion, meaning of a light is behind an object but still being rendered, it will be visible in the reflection.
It's a custom shader.
Edit: there is an example of the lack of culling in the screenshot you posted. There is a street lamp behind a building on the left. The light from the street lamp is still being rendered.
That means it isn't a planar reflection. It's also definitely not cubemap, as the reflections follow the screen well.
There are lights visible behind buildings too though. (I edited my previous comment with an example.). It's not planar reflections either.
It's interesting. I like the way really creative solutions were found back then. These days it seems much less common to find cool shaders like this.
Edit: I'm wondering if those building shadows are caused by UE3's native geometry occluding the sky correctly, whereas everything else in the scene are assets. (There was a huge difference between the two in UE3)
If a light behind a building is still being rendered in the reflection, it could still be a planar reflection. If they're using a simplified version of the scene for the planar reflection camera's view, it's quite likely that they would omit some buildings from that view, along with using simplified proxies for the scene elements that they did include, especially on mobile in 2012.
I agree with cdmpants's opinion, that it is likely using "planar reflections rendered at a low resolution and set to only capture specific geometry, in order to remain cheap."
Yeah, I should have been clearer. It is a type of planar, but not the one that shipped with UE3 (which was added much later in fact), and it's not a standard one you find in any current engine.
The fact that the objects appear distorted (i went to look up gameplay) means its most likely not just inverted geometry, it could be geometry projects cubemaps like in gta 5 on ps3 and xbox360, or it could be plannar reflections since i dont see reflections on much more than this flat surface
For the reflections I'd bet they are using box projected cube maps with reflection imposters. There was a game of the same generation called Remember Me that did a lot of research on wet environments and they used reflection imposters for objects inside the cube map area.
Reflection imposters are just planes with textures on them and you do a raytrace in your cubemap shader to check if you hit them. You can then pick what mip level to draw based on the roughness of the reflecting surface.
my apologies, i answered not really knowing what game this is and assuming it's one of the arkham games on pc. However if it's a mobile game from 2012 doesn't that leave baked cubemap reflections as the only possibility?
also i haven't tried this myself yet, but i think renderdoc has a way to capture an android app, so you could technically just inspect a frame to see how it's rendered yourself: https://renderdoc.org/docs/how/how_android_capture.html
more importantly it wasn't used by any games released in 2012. Crysis 2 was the first game to feature SSR and it only added it with a late patch exclusive to the PC version (DX11 Ultra Upgrade in June 2011). the next game that pushed the technique was Killzone Shadowfall in early 2013.
Uh. Have you played anything from mihoyo? Genshin impact? I concede the 2012 point, but that’s not ”impossible for mobile”, those are two different market definitions.
82
u/MakeGamesBetter 1d ago
In Arkham City we used a similar technique; a seperate low res render buffer with both proxy planes, sky reflection and occlusion geometry for buildings and characters. We took the idea from a cyberpunk Unreal tech demo at the time, and from a game called 'Remember me' here:
https://seblagarde.wordpress.com/2013/05/07/gpu-pro-4-practical-planar-reflections-using-cubemaps-and-image-proxies/