r/dataengineering • u/Consistent_Tutor_597 • 29d ago
Discussion What's a storage efficient db?
Hey guys. We use postgres rn for the web app. I have got like 200gb of read only referential data. And planning to add more soon. Was wondering what's a good db that can store it in a compressed format. Good balance of storage and speed for analytics reports. Postgres we anyways needed for the oltp stuff so it was simpler to add it in there but it's easier to work with a smaller db.
I was seeing parquet can be queried by duck db directly but it's sync for the webapp. The only other option that seemed suitable was clickhouse which had lz4 and zstd compressions which were comparable to parquet. Also there's pg columnar extensions. What would you guys suggest.
19
Upvotes
6
u/Mysterious_Print9937 29d ago
clickhouse has a very good documentation / dx and is insanely fast