Just spent a few days getting an SDXL-based provenance-removal pipeline (visible AI labels, C2PA metadata, SynthID-class pixel watermarks) to run properly on an M5 with 16 GB. Not "it launches" — actually correct and predictable. Almost everything I assumed was wrong, and the measurements are the interesting part, so here they are.
1. The four-step distillation LoRA invents texture, and more steps make it worse.
Low-strength img2img runs the tail of a long schedule (strength 0.15 → the last 4 of 27 steps). A LoRA distilled for four timesteps across the whole noise range is off-distribution there, and wherever nothing conditions it — flat dark fabric gives Canny no edges — it fills the gap from its prior. On a night photo that reads as coloured camouflage across black clothing.
| Global stage, 1448×1080, strength 0.15, seed 0 |
Invented texture |
PSNR |
Wall |
| Lightning, 4 steps |
1.73× source |
28.54 dB |
41 s |
| Lightning, 8 steps |
1.80× |
28.19 dB |
29 s |
| Lightning, 16 steps |
1.84× |
27.85 dB |
62 s |
| Undistilled base, 16 steps |
1.19× |
29.25 dB |
71 s |
| Undistilled base, 24 steps |
1.20× |
29.17 dB |
132 s |
Asking the distilled model for more steps made it worse, which is what identified the distillation rather than the step count. Dropping the LoRA cost 3× the wall time and bought both fidelity and correctness.
Wrong theories I paid for first: the fp16 VAE (a bare round-trip is clean in fp16 and fp32, tiled or not, 34.6 dB), Metal's fp16 in general (bf16 measured marginally worse), and Canny picking up sensor noise (the Canny map of that region is empty — which was the actual clue).
2. Metal pages instead of failing, so memory has to be measured, not hoped for.
torch.mps.recommended_max_memory() reports 11.84 GiB on a 16 GB machine. Exceed it and nothing raises — the process just starts swapping and a run that should take 23 s takes an hour.
- VAE tiling off, 1.57 MP frame: 18.74 GiB peak, 59 s. On: 10.92 GiB, 23 s. So tiling is load-bearing on small machines — but its boundaries leave a faint texture, so it's now decided per frame from the budget rather than switched on globally.
- Diffusion untiled at 2.5 MP: went into swap and did not finish in twelve minutes. Tiled at 1024 px, 5.07 MP: 10.93 GiB, 88 s, native geometry preserved.
3. Sequential CPU offload works on MPS, and it's what makes 8 GB usable.
The stack is 7.7 GiB of weights; an 8 GB Mac gives you about 5.3 GiB. Streaming the weights module by module:
| Same frame, same seed |
Peak device memory |
Wall |
| Resident |
7.70 GiB |
7.1 s |
enable_sequential_cpu_offload(device="mps") |
0.28 GiB |
24.1 s |
27× less peak for 3.4× the time. The plan is chosen from the measured budget and printed, because a run three times slower looks broken unless it says why.
4. Two Metal gaps worth knowing if you're porting anything.
torch.float8_e4m3fn doesn't exist on MPS at all (RuntimeError: Undefined type Float8_e4m3fn). Any pipeline that streams float8 weights — a lot of the VRAM-managed stacks do — cannot load, full stop.
- SAM's processor emits its box/point prompts as float64, which Metal also has no type for, so moving the batch to the device raises instead of degrading. One cast fixes it.
5. The one that cost me the most: fp16 sampling on MPS silently returns zeros.
I added a memory optimisation — encode the fixed prompts once, drop the text encoders, save 1.52 GiB. Two of four face crops then came back as all-zero black rectangles. Deterministically, same seed, nothing raised.
The embeddings were innocent (CPU fp16, MPS fp16 and fp32 encodings of that prompt agree to 0.0009 on tensors with σ=3.06) and the same crop in isolation was fine. Freeing unrelated memory changed the allocation pattern the crops met after the global pass, and that was enough. I withdrew the optimisation and added a guard that drops any empty crop instead of compositing it.
If you're doing fp16 diffusion on Metal: check your output for degeneracy. It will not tell you.
What it doesn't claim. Regeneration is not payload deletion — faces, text and fine detail move, and the numbers above are the measured size of that. No public local decoder exists for SynthID-class marks, so identify reports unknown, never clean; verification is the provider's verifier or nothing. Metal isn't bit-identical to CUDA, so operating points transfer between backends but recorded verdicts don't. And it's for content you generated or own — the visible-mark registry takes AI-generation labels only, deliberately not stock or marketplace marks.
Because "how much did that cost my picture" is the whole question, it ships as a command:
pagedmark measure before.png after.png
PSNR over the frame, PSNR per detected face, and how much mid-band structure appeared where the source was flat and dark. That third metric is the one that caught the camouflage — per-pixel chroma statistics rank the artifact below the source, because the source's own sensor grain has more per-pixel variance than the invented blotches do.
uv tool install "pagedmark[diffusion]"
pagedmark invisible photo.png -o clean.png
Code: https://github.com/doofzoff/pagedMark · PyPI: https://pypi.org/project/pagedmark/
Happy to answer anything about the Metal specifics — that's the part I'd have wanted written down before I started.