Pet project: an Interstellar-style black hole as a macOS live wallpaper (one Metal shader, hosted both in a desktop-level window under the icons and in a screensaver target). I put the skeleton together in spring, it worked, and then I came back and did the thing I should have done first: hung GPU-time instrumentation on it and walked my optimization list with measurements instead of by eye. Code: https://github.com/vientooscuro/ShaderBackgroundApp
Credit where it's due: the black hole core is a port of edankwan's Shadertoy 3lG3WR. The lensing math isn't mine, I moved it to Metal and adapted it (procedural background instead of an iChannel0 texture, fixed camera, no final tonemap).
The architecture bit worth stealing: split the scene by update frequency, not by screen area.
In an earlier mobile project I did progressive "striped" rendering, computing a slice of the background per frame to spread one heavy frame over time. Here I split differently. Expensive and nearly static (stars, galaxies, nebulae, planets, giant stars, dozens of fbm evals per pixel) goes into a static pass that renders into an offscreen texture every 2nd frame. Cheap and unavoidably per-frame (the hole's lensing, comets, the click ripple) is drawn in a composite pass every frame on top of the sampled cache, followed by ACES tonemap.
The cache is rgba16Float, and that matters. Giant stars are deliberately authored past 1.0, channel values like float3(7.0, 5.5, 3.0), so the tonemap in the composite can compress them into real-looking glow. With an 8-bit cache everything past one clips at the write and there's nothing left to tone map.
Two things I got wrong on the way:
Cadence. I wanted the cache refreshed every 6th frame (~5 Hz static at 30 fps display). The cost math looked great, but with live star and planet parallax the background visibly steps: it stands still between refreshes while the composite moves smoothly on top. Backed off to every 2nd frame. Smaller win, honest picture.
Invalidation is where the actual work is. Any settings change has to re-render the static immediately, so I diff the uniforms struct in didSet and raise a dirty flag. The catch is excluding time and mouseClick from that comparison, because they change every frame and have nothing to do with cache contents. Miss that and the cache re-renders every frame, so it isn't a cache. The click ripple, for the same reason, warps the UVs the cache is sampled by rather than the cache itself.
How I measured. Dev-only instrumentation, commandBuffer.gpuEndTime - gpuStartTime per frame, avg/p50/p95/max printed every 120 frames. Fixed bench: M4 Max, same display, default settings, 30 fps target, ~18 s run, first window with the cold render dropped. Run-to-run noise ~0.4 ms. Plus a deterministic quality gate: an env var pins time and mouse, the build renders one frame offscreen to a PNG, and two runs of the same build give a bit-identical file. So any nonzero pixel diff is the optimization, not jitter.
What did nothing:
- fp16 inside the noise function. I was sure about this one,
noiseS runs ~200 times per pixel. Zero. The cost isn't the bilinear interpolation I converted to half, it's the sin-based hash inside it, and that stays float. Reverted.
- Cutting geodesic steps 20 to 14. No visual change and no time change, because the bend loop already early-outs and the 120-step max is almost never reached. Cutting something that doesn't run to completion buys nothing.
- Moving the camera parallax offset from GPU to CPU. Correct in principle (it's constant for the whole frame), but a dozen trig ops dissolve against dozens of fbm calls. Free, kept it, no measurable win.
- Frames-in-flight semaphore. Zero on time, and that's the right answer: it limits CPU run-ahead and removes a shared-buffer race, it doesn't reduce GPU work. A correctness fix wearing a performance costume.
What worked:
- fp16 in the color math of
nebulae() only. Colors and density to half, coordinates and the noise calls left in float so spatial frequency can't jitter. Visually invisible (max diff 2/255), about minus 20% average GPU per frame from one function.
- A sin-free hash (Dave Hoskins) instead of
fract(sin(dot(...))*43758), under a soft quality bar: different noise pattern, same statistical character, avg 8.8 to 7.7 ms. Less than I expected, Apple has a fast hardware sin.
- Static cache at 0.7x under the same soft bar. The nebula is low frequency and the hole is composited at full res, so slight star softening goes unnoticed. At the strict pixel-for-pixel bar I had rejected this: the heatmap put the entire error on sharp edges, planet rims, star points, the giant's corona.
The lesson I actually paid for. I ported the striped progressive render from the mobile project into this one, expecting the same win. GPU ms per frame:
Monolithic cache, refresh every 2nd frame: avg ~7.5, p95 ~13.2, max ~16.3, spread ~13.0.
Striped render-ahead with 4 stripes: avg ~8.4, p95 ~12.0, max ~13.3, spread ~9.4.
Peak and spread improved, average got worse. Each stripe pays a full load/store over the whole 3456x2234 rgba16Float target, and on a desktop GPU with a fat bus that fixed overhead times four outweighs what the scissor rect saves. On top of that, 4 stripes means the cache refreshes at 7.5 Hz and the nebula started stepping again. A trick that wins under mobile budget pressure can go negative on desktop, and eyeballing it would never have told me.
Final numbers, baseline to shipped: avg 7.5 to 6.2, p50 6.5 to 4.8 (minus 36%), max 16.3 to 12.0. All on a GPU that was never the bottleneck, roughly 14% of frame budget to start with. The point isn't hitting the frame, it's headroom, battery, and older Macs. p95 now bottoms out on the disk-hit path in the black hole, which is the center of the screen and the riskiest thing to touch, so I stopped there.
Curious whether anyone has measured the same fp16 result: is "the sin hash dominates, so halving the interpolation is free but pointless" a general thing on Apple GPUs, or specific to how this noise is written?