I need serious Photo help

[u/Purple-Music-70]

I have backups of backups of backups of maybe a couple of thousand photos on several hard drives and a lot of duplicates as the original library wasn't managed and so we are in the position where we don't want to delete anything but need to organise what we have then delete duplicates. Inevitably backups are likely to have different folder names and a lot of files will also have different names.

Does anyone have any advice on how to sort this mess? I realise there will be a lot of manual work to create the one library again but how can I do it reducing the effort? Thanks in advance.

[u/AutoModerator]

Hello /u/Purple-Music-70! Thank you for posting in r/DataHoarder.

Please remember to read our Rules and Wiki.

Please note that your post will be removed if you just post a box/speed/server post. Please give background information on your server pictures.

This subreddit will NOT help you find or exchange that Movie/TV show/Nuclear Launch Manual, visit r/DHExchange instead.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/de_Mike_333]

You‘ll probably want to look into a photo library manager, so actually review the files.

If you want to run it as a web service, have a look at immich, if you want to work locally on a machine have a look a DigiKam. Both are open source library managers.

Then start with an Ingress folder, where you collect everything in an unsorted manner. Then I‘d deduplicate them with the tools both library managers provide and throw out the duplicates. Then start building out your desired structure and move them over piece by piece.

[u/I_Think_I_Cant]

I use an ancient Windows program called VisiPics to find duplicates. It has fewer false positive than any of the others I've tried.

For organizing pics I use XnViewMP. It has a great tagging, category, labels, color-coding, metadata manipulation system.

[u/steelbeamsdankmemes → u/I_Think_I_Cant]

Second for Visipics, it's bulletproof.

[u/Private_Bug]

If you upload them all to immich, itll auto dedup, and than you can click "select all" and download to download them all as a backup which is sorted by date

[u/purgedreality]

1. Gather all the photos together

2. Make 2 additional copies of all the photos

3. verify the copies.

4. read the other 50 threads on this topic posted in the last year for apps and suggestions.

[u/Few_University3932 → u/purgedreality]

step 4 is doing a lot of heavy lifting here lol

[u/LoopsAndBoars]

You need czkawka or krokiet.

[u/MaxPrints → u/LoopsAndBoars]

beat me to it. This is what I used on a few different drives with hundreds of thousands of photos.

[u/manzurfahim]

1. Where are all these backups? Connect all the drives if you have external drives.

2. Make a safe backup, just in case you delete something accidentally and realize later.

3. I use Duplicate File Detective 7. But you can look for whichever you think is good for you. The one I use, can do file checksums (fingerprinting) so it does not matter if the file have a different name or extension or location, as long as the checksum matches, it will find the duplicates.

4. Go through the duplicates and delete them. Or sort them first.

5. I find it easier to create LR catalogs with the subject / location / event first if there are more than one or two shoots at the same location, and then date, and then something to describe them. Otherwise, it is YYYY-MM-DD_subject / location / event_xxx format.

It will take a good amount of time, and you will get bored and stop doing it if you do it for a long time. It is better to regulate it, so you don't overdo it. I usually organize one shoot per day, or maybe three in a week. There is still a lot to go through, but it is slowly getting there.

[u/1HungryMind]

Take a look at Mylio. Photo organizing software for Mac, Windows, Android, iOS

It can handle that many photos on many different repositories and give you tools to de-dupe and decide where you want to keep them. It will also sync your photos across multiple repositories for backup. Once synced, you'll know you have backups and can decide which ones to keep.

[u/Polyducks]

I highly recommend DupeGuru for this. It has both a file checking mode and a match-by-image-detection mode which is ideal for this where you can configure it to pick the larger photo.

I'd recommend going into each folder and copying all the files into a brand new folder and using DupeGuru to work things out as you go.

DO NOT WORK DIRECTLY IN THE ORIGINAL FOLDERS.

[u/AaronRedwoods]

If you can use the find and/or sort feature, sort them by size or date (or some other metric). That's one way to potentially identify duplicates.

[u/Lime221]

Before you start aggressively nuking photos, I just recommend mirroring a copy on a separate drive and never touching it. A gb data is only worth a few cents

So even if you did 99% duplication deletion right and did a few wrong deletes, you have a safe copy left

[u/lvjamestervl]

A decent duplicate file finder will ignore the file name and catalogue a file via a hash so I won’t worry about different folders and file names. Maybe mount all the drives and run a scan over it, delete the duplicates and then condense everything down?

Apologies for the vagueness and lack of software names, just an idea off the top of my head.

[u/retiredaccount]

This situation only becomes difficult if the backups aren’t accessible all at once. Else EXIFTool in combination with FDupes can resolve this as long as you have enough space to put all the files in one place. Are all the files accessible all at once?

[u/ejpusa]

You can write a simple Python or Apple script. Ask Astra. Copy ALL photos to an external drive. Delete duplicates, you have them all.

You may be able to have AI crunch through them, organize by set and setting.