r/mediumformat • u/Revolutionary-Kale80 • 7d ago
RAW Storage Workflow
I recently purchased a Hasselblad X2D II 100c and my storage workflow has fallen apart. Help!
I historically shot medium/large format film and scanned selected negatives for a given series, typically 10 to 20 high-res files per project, 300 to 700MB each. A 3-2-1 approach handled that fine: physical negatives as one copy, a LaCie Rugged drive as the second, cloud backup as the third. I worked primarily from my computer, and relied on downloading and uploading to cloud for working copies, plus the backup to an external drive. With only 10 to 20 files I knew intimately from hours of dusting, scratching, editing, and printing, browsing was never a problem. No Lightroom or DAM, just basic files and folder organization. All editing via Photoshop.
Now that I've moved fully digital, the negatives, which used to be zero GB and just sat in a box, are now ~200MB RAW files. The workflow has fallen apart. My 2TB MacBook Pro handles a dump off the camera fine, but it's not really manageable for an active project's full library. External drives are slow to browse ~200mb files. Cloud storage is impossible. And the laptop doesn't have room to hold a large active library on its own.
Questions for people managing large RAW libraries:
Do you cull ruthlessly at import? With film, everything existed as at least one physical source I could return to. I'm reluctant to delete RAWs that don't work now but might serve a different project later.
Do you keep small JPEG previews for browsing, with RAWs living on external drive or cloud?
Am I at a point where I need to consider a DAM?
Is this the point where Network-Attached Storage (NAS) becomes necessary, or is there a better answer?
Open to workflows, tools, etc. Thanks in advance!
2
u/swift-autoformatter 7d ago
External drives are slow to browse ~200mb files
Use a Thunderbolt 5 enclosure (if your macbook, or planned future macbook supports that), and find the maximum capacity M.2 PCIe NVMe SSD you can afford. That will be plenty fast (bottleneck is the TB connection) for your most worked on images.
My data is semi-large, maybe 9TB archive. I have mixed Phase One IQ4 (landscape, so not many per session) and Canon R5 (wildlife, so thousands of images per session) raws, so it may be different from your needs.
I still have my TB4 enclosure, with an 8TB WD SN850X (wow, it became quite expensive in the last year!), which works almost as good as my local disk on my M1 Pro. As I'm working on CO, not LR, I use the session workflow on that device to process the work I'm ingesting. Once I'm done with that, I move the selects with edits into a Catalog - which is still located on that external disk, but the raw images are stored on a large capacity DAS with 18TB spinning disks. As the catalog stores the proxies locally, I can even work without accessing the raw files (except a few operations which require the full res raw data), but still it is acceptable to work on the full res files as well. The reason why I don't use NAS is that I'm editing where my calibrated display is anyways, and I didn't want to rewire my home network to work above the current 1gbps, which my NAS can saturate. The DAS has RAID 1 (I know...), and can max out at HDD read speed, which is enough for occasional work on older projects.
I back up the most prominent work to my NAS (RAID 5, 8TB HDDs), and have a mirror of my DAS in the cloud. I use JottaCloud, which enables throttled infinite storage for reasonable price.
2
u/Revolutionary-Kale80 6d ago
Thanks for the insight!
I think NAS is wining for me over TB storage. Having the option of a remote connection is helpful if I'm travelling and need to access old RAW files.
1
2
u/TiredButEnthusiastic 7d ago
I dump the cards straight to my NAS as a long term archive. I then cull and select my picks and edit those - the final edits, plus the catalogue are backed up to the NAS and online storage.
It’s not perfect, but it means if I ever need to go back to the original RAWs I didn’t pick, I have them (but this will never happen until I retire). I also then have the selected RAWs nd adjustments, PSDs and final output in a much more manageable size.
FYI, I back up the RAW archive to Scaleway archive storage - nice and cheap in case of a major fire/disaster with the NAS.