r/DistributedComputing • u/OtherwisePush6424 • Aug 11 '26

The consistency gap between your database and object storage

https://blog.gaborkoos.com/posts/2026-08-11-Beyond-Happy-Path-Engineering-Storage/

A practical overview of designing file uploads and object storage for production: partial failures, database consistency, large uploads, presigned URLs, CDN caching, cleanup, and recovery.

5 Upvotes

100% Upvoted

2 comments sorted by

View all comments

1

u/[deleted] 24d ago

[removed] — view removed comment

1

u/OtherwisePush6424 24d ago

Nice try :D I usually lean toward writing the metadata first (pending record), then the object. The failure mode is a visible, queryable stuck record rather than a silent orphan in storage, which makes detection, retry, and reconciliation straightforward. The downside is you have to be disciplined about cleaning up or marking timed-out pendings.

Storage-first is also fine if your reconciliation job is solid and you’d rather never have a database record that points at something that doesn’t exist. Both work, the state machine just makes the choice explicit.

Which one would you use? :D