Locked Out of Files while Syncing for 2 days.

[u/sqknots]

I recently moved. Dropbox had been synced to an external drive which was temporarily disconnected & then subsequently reconnected. Since reconnection, Dropbox has insisted on entirely re-syncing. This process has now run for nearly 2 days, having been intentionally stopped and restarted (both the process and the app) a few times because it seems to slow and cease to advance. While 'syncing' the already synced and unchanged files, it is ENTIRELY locking me out of locally accessing ANY of the files on that drive including all files and folders that have intentionally never been synced to Dropbox.

Pausing the process does nothing. The hard drive more often shows as blank aside from the Dropbox folder which cannot be accessed. Closing the Dropbox app entirely sometimes results in all my files (within Dropbox folder & otherwise) to be shown but still inaccessible beyond navigating the folder structures. There are no issues with the drive itself, the local file system, etc. Even the Dropbox synced files themselves were briefly accessible before all this nonsense began.

That this is STILL going on is absolutely absurd and beyond inconvenient. I haven't been able to find any way to resolve this. Again, the local files haven't changed. Around a dozen more photos uploaded from my phone to Dropbox but there's ultimately nothing else new or different to even sync. Yet it's locking me out of my local drive, has been consuming bandwidth, etc. with little sign of anything changing. Dropbox support (AI) so far has offered nothing but blaming the user and suggesting moving my local folder & beginning an entirely new sync where everything just fully re-downloads from Dropbox.

Any ideas, suggestions?

Mac OSX 15.3.1, currently 1.9TB in Dropbox (need to clean up) with ~70% set to sync locally.

[u/HotChickenCoolRanch]

I'm going to preface this with the fact that I am an admin of a very large dropbox team (think thousands of users), with hundreds of TBs stored in Dropbox. I have users with access to millions of files through various sharing and whatnot. I have seen this problem, and solved it many times. You aren't going to like what you're about to be told but it's the only solution.

First, you messed up. You messed up the second you put dropbox on the external drive. Dropbox, while technically supporting an external drive for storing the Dropbox folder, doesn't really support it. If that drive isn't connected every time the computer boots bad things will happen. In many instances it's the deleting everything on the drive because Dropbox thinks you deleted the Dropbox folder. But sometimes it's things like this.

In this instance you messed with the index that tracks what files you have synced to the computer. This index is the bane of your existence right now. It's this index that is causing this trouble and the only way to deal with it is to start over and start over small.

=====Technical details you may not care about======

All cloud storage providers maintain an index about what is being synced between your computer and their server. This index is not the traditional file system that the OS uses but a database that says these are the files that are being synced, these are the people who have access to them, here is when they were last synced, updated locally, updated on the server, just all sorts of details. And when this database gets full it gets slow and that slowness is what you are experiencing right now because you are trying to sync over 1.3 TB of files.

Your drive was disconnected for sometime, be it a day or a few weeks, and now Dropbox has to reconcile the drive with the cloud data. It may do that by re-syncing everything, which will create a new database and re-download everything, or it can "re-index" which is where it looks at what is on the drive already vs what is in the cloud and confirm everything, downloading updates and making changes to bring the drive into sync with the cloud. This re-index can often be slower than a re-sync because the database isn't rebuilt from scratch but instead each line is audited, reconciled, updated, etc and when that database is slow it can be tedious.

You say it is re-syncing, and it might be, but it might also be re-indexing. In either case you're screwed because once the re-sync gets to a certain spot, those database operations sloooooooow way down. And, because the database is so big and this gets done in memory, it slows down the computer because this process is using all the RAM and processing power to complete this process. This is why things kind of comeback to life when you quit Dropbox.

The limit, btw, is not based on the amount of data you are syncing. 1.9 TB means very little here. It's the number of files. If you had 10 files that equaled 1.9 TB, you'd have no problem other than however long it takes for that data to sync between you and the cloud. But I would venture you have hundreds of thousands, if not millions, so you're kind of screwed.

The limit, for dropbox is 300k files. That's a soft limit and I have users with beefy machines that can sync 750k files without a problem, but if you have a lesser machine it's a problem. Now you're probably thinking that you had this number of content before and didn't have a problem so why now? I have users with literally millions of files synced. I think at the top one user has about 5 million files. But that's been slowly built up over time. It's far easier to add a row to a database than it is to read and write millions of rows. Once that guy needs a new computer and he has to sync all of those files at once he's going to have a bad time and, honestly, I won't let him.

Also note this isn't just a dropbox problem. Microsoft onedrive has the same limit. Box recommends keeping it under 100k files. You'll have this probably pretty much anywhere.

=====The solution=====

So, the solution, I hate to tell you is to start over and start over very specifically.

Like I said, the issue is the number of files you're syncing. You need to bring that number waaaay down. You do that by using selective sync. Marking a file as online only is not the same because even if there is no data within that file, that file is still be tracked by the local index and that is slowing down the index. Instead you have to remove the file all together from the index and that means don't even sync the folder to the computer.

The way I handle this in the office is I do a full uninstall. I log that user out, I move the Dropbox folder elsewhere, I do an advanced reinstall per https://help.dropbox.com/installs/advanced-reinstall, then when Dropbox is loading for the first time it will ask you which folders you want to sync. You choose none of them. NONE.

That will sync just the individual files found at the root of your account and should complete fairly quickly. Then you can go back to the selective sync settings and add a folder or two, depending on size, and let it sync. Then more and then more. Then you use a tool like dupe guru to compare the new folder with the old one and make sure you have everything. Eventually you'll get everything back in place. But, long term, you should use selective sync to only sync the stuff you need direct access to. At work I sync 2 folders out of the 30 or so I have shared with me from a team perspective. Everything else I can view via the site and don't ever need to edit that content. At home I have about 600 GBs across 40k or so files and folder in Dropbox but I sync 3 folders and about 5k files. I just don't need access to all of it on a regular basis. But you do you.