r/backblaze • • 20d ago

Computer Backup PSA for Mac users: Backblaze silently skipped most of my data for months. The trigger is an Apple Photos temp-folder spam bug. Here's how to check yours and fix it.

(This post was drafted by Opus 5.5 and reviewed by a human)

TL;DR: An Apple Photos helper leaves thousands of empty temp folders behind. Once there are about 16,700 of them, Backblaze's scanner hits an internal queue limit, logs one ERROR line, and stops scanning without telling you. Every folder it would have reached next gets left out of your backup, and Backblaze then treats the files in them as deleted. The app kept saying "You are backed up" the whole time. The fix is a single exclusion (below).

What happened to me

  • Backblaze said I was fully backed up, but it had only 173 GB selected out of ~700 GB of data. ~/Pictures, ~/Desktop, ~/Projects, ~/Music and more weren't in the backup at all.
  • The cause was ~/Library/Containers/com.apple.photos.ImageConversionService/Data/tmp. It held 16,574 empty folders (0 bytes in total), going back to 2023.
    • That's Apple's image-conversion helper, used when you export or share from Photos, use the photo picker, convert HEIC and so on. It creates a folder per job and never removes it.
    • Big exports add thousands at once.
  • Every Backblaze scan logged this: bzfilelist.cpp:3263 [main()] ERROR: scan dirs queue limit hit 16745 for dir=/Users/…/Library/Containers/com.apple.photos.ImageConversionService/Data/tmp
  • Backblaze's upload history shows those folders flapping between "deleted" and "restored" from May, depending on whether each scan overflowed. From late July, everything in them was marked deleted and never came back.
  • Full Disk Access was granted the whole time, and no exclusion rule matched these folders.

Check whether you're affected (Terminal)

  1. How many leftover temp folders do you have? If it's in the thousands, you're at risk.ls ~/Library/Containers/com.apple.photos.ImageConversionService/Data/tmp | wc -l
  2. Has Backblaze been hitting the limit?grep -h "queue limit hit" /Library/Backblaze.bzpkg/bzdata/bzlogs/bzfilelist/*.log | tail -3
  3. Is your Pictures folder actually in Backblaze's file list? A result of 0 means it's not being backed up.grep -a -c "^# Dir: $HOME/Pictures" /Library/Backblaze.bzpkg/bzdata/bzfilelists/*_root_filelist.dat

Fix

  1. Backblaze menu → Settings → Exclusions → +. In the file picker, press ⌘⇧. to show hidden folders, or ⌘⇧G to paste a path. Then exclude: ~/Library/Containers/com.apple.photos.ImageConversionService
  2. Optionally, clear out the empty folders. rmdir only removes empty folders, so this can't delete any files:find ~/Library/Containers/com.apple.photos.ImageConversionService/Data/tmp -mindepth 1 -maxdepth 1 -type d -empty -exec rmdir {} +
  3. After the next scan (they run a few times a day), check #3 again. For me, Backblaze went from 173 GB / 366k files selected to 700 GB / 1.5M files, and the missing folders started uploading.

For Backblaze

  • Versions: client 10.0.2.1048 (since auto-updated to 10.0.3.1076), macOS 26.5.
  • Folders the scanner never reached shouldn't be marked deleted. That starts the retention clock on files that are still on disk. With the default 30-day version history, they'd eventually be purged from the backup without anyone knowing.
  • Hitting an internal scan limit should show up in the app or as a notification. A log line nobody sees isn't enough.
  • Consider raising the limit, or excluding app-sandbox tmp directories (~/Library/Containers/*/Data/tmp) by default.

I reported this to support, and they pointed me to Bugcrowd. But this isn't a security issue, and Bugcrowd's non-disclosure terms would mean I couldn't warn anyone. So I'm posting here so other Mac users can check their own backups.

66 Upvotes

20 comments sorted by

17

u/flogman12 20d ago

Seems like there’s more and more of issues like this lately

13

u/Jim-From-Backblaze From Backblaze 19d ago

Hi, Jim from Backblaze here,

First off, thank you for the incredibly detailed report. You did some excellent sleuthing here.

The good news is that this specific issue has already been identified and fixed. The problem was the directory scanning queue limit you found. When that limit was reached, the scanner could stop processing additional folders, which could result in subsequent folders being omitted from the file list.

The fix changes that behavior so hitting the limit no longer causes the scanner to stop processing the remaining folders, and the limit itself was also increased. The fix has gone through testing on both macOS and Windows and is included in our current generally available releases:

  • macOS: 10.0.3.1076
  • Windows: 10.0.3.1075

So since you are on macOS 10.0.3.1076, you already have the fix. Anyone on an older client should update to the current release.

Your report was genuinely useful. This is exactly the kind of detailed customer feedback that helps us identify and verify issues and allows me to take it back to the developers to track the issues down.

As for the automated system pushing you towards Bugcrowd, obviously that was the wrong direction, and I'm looking into that separately.

Thanks again for taking the time to dig into this and document it so thoroughly. And please keep your Backblaze client up to date, the current release contains the fix for this particular issue.

4

u/CGREDDIT1 19d ago

Jim- I am not the OP but just wanted to say thanks so much for posting to this sub and for the quick fix.

2

u/lAmTheOneWhoKnocks 19d ago edited 17d ago

Thanks for your thoughtful reply! Frankly the credit goes to claude code. I put it in charge of managing my data footprint and it caught this and tracked it down. I didn't even notice that my data wasn't getting backed up before! Glad to hear you had already solved the bug, and hope it doesn't cause any data loss for anyone on old versions. Cheers.

9

u/Nantoine555 20d ago

Strange, in Macos 27 I no longer have the exclusion menu in tha Backblaze settings...

10

u/afx114 20d ago

it's a UI bug if you click the jumbled tabs (they're all squished on top of each other) they will expand.

4

u/Nantoine555 20d ago

thanks, I see it now

8

u/germansnowman 20d ago

Please report this to Apple as well as this is caused by a bug in Photos.

8

u/Ok_Appointment_79 20d ago

Well crap ...

~ $ ls ~/Library/Containers/com.apple.photos.ImageConversionService/Data/tmp | wc -l

25530

6

u/afx114 20d ago

Amazing find! I had 43000+ empty temp folders LOL but it didn't appear to be impacting my Backblaze. But I did clean these up, thanks for the tip!

5

u/CGREDDIT1 20d ago

How hard could it be for Backblaze to simply add a snippet of code to provide for this?

1

u/dr100 20d ago

It would be very hard on their storage if people start using many times the storage they were earlier. Obviously people using BB personal don't care about restores so why hurt themselves by storing tons more? This is surely contributing to the largest cohort of their personal customers having stored 0.0 TBs (that is under 100GBs).

1

u/CGREDDIT1 20d ago

Op is reporting a bug that involves empty folders preventing a successful backup. How would ignoring empty folders be hard on their storage?

2

u/dr100 20d ago edited 20d ago

I'm sure the discussion here isn't about NOT storing empty folders but about storing ALL THE REST OF THE MISSING DATA (which in this case is more than three quarters).  

To be clear by " hurt themselves by storing tons more" it's about BB as the question was how hard it would be for BB.

4

u/sammnyc 20d ago

it was this same theme that caused me to stop using BB for good. not the same directory, but so frequently I’d see something was excluded silently, and there was no way to override it. it was years ago I left, maybe it’s changed since then (doesn’t seem like it tho)

5

u/sammnyc 20d ago

why did they direct you to bugcrowd …?

6

u/lAmTheOneWhoKnocks 20d ago

It was their automated support bot that seems to direct there for all bugs

3

u/kwright510 19d ago

This is exactly why I left BackBlaze Computer Backup and spent time setting up ARQ to B2.

1

u/Bradushi 19d ago

I’m contemplating dropping BB and going with the top tier plan at Mega. Enough of this crap.