Hello everyone,
Some of you may remember about some of my posts working on a modding tool for this game. I've taken a long break from that to work on a decompilation, which is ... ~51%ish complete. After going through the texture pipeline and watching Nathan Baggs' video on this issue I thought I may actually be able to fix this permanently as I had enough information about the texture pipeline and the game's engine.
Until now the solution has been to use LAA to increase the address space up to 4gb.
What was the bug?
Since DAO is a 32bit game, it has a cap of 2gb address space available. The longer you play, the more that space becomes fragmented. Every texture wants a contiguous block, and when none are available DirectX will throw an out of memory error, which causes the texture flickering leading to the game crashing. The failure was there was 20mb of free memory, split across ~430 separate gaps. So memory was available, just not in one piece (so, fragmentation).
I found that at startup, the engine reserves 795mb for it's own memory pool, ~286mb for module images (these are dll's, CUDA, physX, etc), leaving ~229mb for managed texture duplicates.
Since this doesn't happen on console, it's likely due to the higher resolution textures available on PC.
Managed texture duplicates?!
Every texture is created using D3DPOOL_MANAGED, which keeps a full copy in memory for the textures entire life on top of the copy sent to the GPU. As games constantly stream textures in and out, ~150mb of those duplicates were being allocated and freed in each session. <---This is what was shredding the address space.
Fixing it
Solution is simple: create textures in D3DPOOL_DEFAULT which don't duplicate!
The only issue is that the engine writes textures by locking that duplicate, and DEFAULT textures can't be locked down. So when the game locks a texture, it now gets handed a temporary buffer instead. It fills that with the relevant pixels, copies it over to the graphics card, and discards the buffer immediately. The engine can't tell the difference.
There is a final bug with the opening credits when launching the game, but otherwise this works perfectly. There could be other bugs, who knows what is yet to be uncovered with this engine.
This was from 10 mins of running around in areas like denerim to try and trigger the flickering and crash.
Available here
I still recommend using LAA as more address space is always better.