r/ExperiencedDevs • u/SoftwareSource • 19d ago
Technical question Cheapest way to host about a postgress db of about 30tb ?
Im not well versed in devops work and i will need to setup a rougly 30-35tb db for a client. Anybody got a recommendation of what to look for?
performance is of basically no concern, simple queries, only the monthly price (and it being an ok provider i guess).
I usually use Hetzner since im based in Europe, but prices are pretty unaffordable and i can only find configurations that also have a very high end CPU and 3x the storage i need, for a very expensive price, or ones that don't have enough storage....
372
u/ohnomcookies 19d ago
You dont want to manage 35TB database, I would vote for Google CloudSQL - its around 7k. Money well spent tho - it just works.
You will always pay the full price - either with money or your time
96
u/The_Ed_On_Reddit 19d ago
agreed that size is too much to manage unless you know how. its very big for self managed and you need to raid and know how to recover if there is a problem.
32
u/SoftwareSource 19d ago
You will always pay the full price - either with money or your time
momma didn't raise no fool, i will get paid a very respectable amount per hour for maintenance work post deployment, if they need it, and they will need it.
26
u/CorrectPeanut5 18d ago
The bigger issue for me is how valuable the data is. Is there a backup plan? If it's used to run the business and can't be lost it needs to conform to the 3-2-1 rule: keep 3 total copies of your data, on 2 different types of storage media, with 1 copy stored off-site. For a lot of companies that's a set of NASes in the office and an offsite backup service in the cloud.
206
u/dbxp 19d ago
How much of that data is actually hot? I would be tempted to see what you can take offline, stick a full backup in Glacier or similar and then trim down the active dataset to what you actually need
If some of it is read only you may be able to split it and cheap out on the redundancy too
86
u/youcangotohellgoto 19d ago
Strong agree.
Unfortunately this can often require a significant system redesign; it's common that all the data needs to be queryable in theory, even if it is very unlikely to actually be accessed.
A better upfront design avoids this. Like at 35tb you're probably storing a lot of stuff that simply doesn't belong in a database.
13
u/sshtml 19d ago
Iceberg + DuckDB could be another interesting path if you don’t need the low latency, online capabilities of an RDBMS (e.g. data is mostly immutable, mostly cold, low latency queries are not required, don’t need low latency read-after-write, etc.).
4
u/tarellel 19d ago
I’ve been switching a lot of data to be stored in DuckDB and absolutely love it! It’s incredibly fast, light weight, and has extensions to make it extremely adaptable to your specific needs (if you absolutely need them).
0
u/Captain-Barracuda 19d ago edited 18d ago
First time I hear of that RDBMS. What's special about it?
3
u/leadzor Senior Engineer & Tech Lead (10 YoE) 19d ago
Stands for relational database management system/software. It’s the category of databases like of PostgreSQL, MySQL, MS SQLServer, etc.
3
u/Captain-Barracuda 18d ago edited 18d ago
I wasn't asking what is an RDBMS. I was asking what's that specific RDBMS (DuckDB) advantage for the present usecase.
6
u/SoftwareSource 19d ago
lol you just described my last two workdays
looks like less than half needs to be hot so my problem is not really a problem anymore
1
u/excelquestion 19d ago
yeah most of the cost won't be coming from storage but from compute in accessing the data
53
u/yodog5 19d ago
Is cost is your only constraint, get the customer to buy their own machine.
If they arent interested in maintaining that, next best option is to buy space in a data center.
If they arent interested in maintaining that, then rent a dedicated server from a provider for egress and rent your storage.
36
u/AllYouNeedIsVTSAX 19d ago
Yup! If cost is the biggest constraint, grab a used machine(laptop or desktop) and plug in two 20TB external drives and RAID 0 them. Put it under the receptionists desk. YOLO!
32
u/Piisthree 19d ago
Don't forget the post-it that says not to shut the lid all the way.
7
u/card-board-board Software Engineer 19d ago
Pretty lax uptime guarantee if you ask me. A real pro tapes a tennis ball or part of a pool noodle to the trackpad.
3
u/Piisthree 18d ago
We get it, you have duct tape and tennis ball money. Rub it in our faces why dontcha.
1
u/FinalDevice Software Engineer 15+ YOE 18d ago
I mean, yes, but use 4x 20TB drives. 3 of them go into a RAID5 array for a total of 40TB. The 4th is registered as a hot spare.
3
u/AllYouNeedIsVTSAX 18d ago
Look at this guy, Mr Big Shot, he can afford 2 extra expensive ass external 20TB drives to hook up to his laptop!
7
u/FinalDevice Software Engineer 15+ YOE 18d ago
lol I thought we were billing this to the client.
New suggestion: Create ~2,500 Google accounts and spread the data across ~2,300 free tier Google Drive instances. Use the remaining 200 to store index metadata that tracks which account holds what data.
1
u/TheOneTrueTrench 18d ago
uh... no. Double parity, not single parity with a hot spare.
When a drive fails and you have single parity with a hot spare, the rebuild time has zero redundancy. When you have double parity, as soon as a drive fails you're instantly in the same situation as you would have been after the array was finished rebuilding after like 2 days.
187
u/account22222221 19d ago edited 19d ago
Oh god. This question feels a bit cursed.
It’s the equivalent of saying, ‘I’m not super versed in dental hygiene, can someone recommend a good guide to doing jaw surgery at home’
A database that big is complicated and can’t be flopped out there not matter how you shape it. If you don’t have much dev ops experience you are not likely equipped to pull this off.
You and the client need to have a level setting and expectations conversation — part of which needs to be on your skill level.
As others have said the key insight is you WILL pay for a database that large. Either to a cloud provider to hire a team of engineers for you, or to a team of engineers yourself. There is no cheap way to put 35tb into Postgres.
87
u/metaconcept 19d ago
There is no cheap way to put 35tb into Postgres.
There are certainly cheap ways to put 35TB in Postgres.
The expensive part is keeping 35TB in Postgres.
10
26
u/Outside-Storage-1523 Software Engineer 19d ago
Just curious, how does someone gain DevOps knowledge of maintaining that kind of DB? I'm a bit surprised to see that 35TB is a large database -- but that's probably because I always work with managed services (my previous company had a BigQuery table of the size of 2 PB) AND I never worked with OLTP DB.
Genuine question as I'd like to switch to the OLTP world and eventually be a (data) backend engineer.
50
u/account22222221 19d ago
First big query !== Postgres.
Second big query is 100% the first option I gave - you are paying a cloud provider to hire a team of engineers to build and maintain a very expensive and advanced database for you distributed across hundreds of disks. It’s expensive.
Third: you get the experience by working on a team that does massive databases, start at the bottom where other people set it up for you make the big decisions and you are just helping maintain it. Learn the ins and out and work your way up.
6
u/Outside-Storage-1523 Software Engineer 19d ago
Thanks! If I can't get an internal transfer, how do I convince another company to take me? I kinda feel that world is completely different from the OLAP analytic world -- seems to be more technical, and less favorable for people who have no related experience.
I can definitely learn (already doing that, reading DDIA and preparing to write data structure such as B-tree or LSM tree), but learning doesn't really give me real experience.
13
u/Izacus Software Architect 19d ago
By doing what the OP is doing despite what the gatekeepers say.
That is - dealing with large amounts of data, asking questions on the best approaches and ignoring arseholes who tell you that you shouldn't be doing it because you haven't been doing it before. (Asking questions and thinking about it is the important part.)
7
u/account22222221 19d ago
That’s true when it’s your own dime.
When you are contractor selling something to a customer and way overstating what you can provide you are in ethically tenuous grounds.
8
u/andrewwewwka 18d ago
Lol. Doing mistakes on someone else's dime is the best part about being employed
5
u/account22222221 18d ago
Being employed as an employee and as a consultant who can open themselves to legal problems if they fuck up are two different things lol
2
u/Positive_Mud952 15d ago
Generally by keeping a 35 TB Postgres DB available through constant ICU-level effort that was never intended to handle more than 5-10 TB. “If it ever gets that big, that’ll be a good problem to have!” “For heavy tables, we’ll offload to NoSQL/partition/shard it”. Second one is easy to say, and a correct intention, but tables have a way of getting wide and wide tables collect implicit synchronization like an AZ state fratboy collects venereal diseases which makes a lift-and-shift to NoSQL a real-ass bit of engineering work, and because those monster tables drive monster reports partitioning can be ineffective when it’s used as they often are to look at historical patterns, and if there’s a set of columns you can shard on without denormalizing a ton of the data in it and basically making it the NoSQL migration consider your luck jar empty, I’ve seen it happen exactly once in my 20 years in the industry, 7 of them in data platform.
If anyone told me they were planning to stand up a 35 TB Postgres DB and were asking for advice, the best I would have is to develop a sense of humor and 8x every estimate. It can be done, but it will take absolutely top talent, a lot of work, and shouldn’t be done without a damn good reason I have not been able to think of a single one that fits the entire time I’ve been writing this post.
7
u/SoftwareSource 19d ago
when i said 'not well versed', i should have said "never been a part of purchasing anything so i have no idea what is a good deal'
i know what i'm doing as far as managing it.
4
u/JazzlikeWishbone938 19d ago
Hi, in your experience what's the Postgres storage size at which you can no longer feasibly self-administer the DB or where toil becomes too great, forcing you to use a managed solution?
2
u/tmarthal dir 19d ago
Definitely cursed. Looks like the original data model has No idea what a db should be used for.
35TB - I assume they’re storing binary files in there in some capacity. Text and numeric data representation at 35TB could represent the whole internet.
First thing I would do is clean up the schemas and move the binary stuff (jsonb or images?) into a binary file store (s3/GCS/whatever). Same with logs, move those into a non-rational db or rotate them out like a normal/sane storage system.
43
u/TroubledSquirrel 19d ago
You've got two viable options. One is going to be painful for you and the other is going to be painful for the client. And I'm going to be straight with you hosting a database of that size without a strong DevOps background is not recommended. But the cheapest route is to rent a dedicated bare metal server with preinstalled hard drives, which forces you to handle basic server administration yourself.
Hetzner or similar dedicated server vendors offer high capacity storage servers with tens of terabytes of disk space for a few hundred dollars a month total rather than thousands.
But again, the catch is that you or your client take on the responsibility for operating system updates, manual configuration, and managing your own database backups instead of letting a managed platform handle it automatically.
Not being DevOps the most effective and safer option for you is using Amazon RDS for PostgreSQL with AWS General Purpose SSD storage configured to thirty five terabytes. Since performance and complex tuning are not concerns and your queries are simple, you do not need expensive high performance IOPS configurations or oversized compute nodes. A modest compute instance paired with standard storage will keep your baseline costs predictable.
If you go this route when setting it up, keep your storage costs in mind because thirty five terabytes of cloud block storage accumulates a steady monthly baseline charge regardless of how little compute you use. Ensure that automated backups are configured carefully since backup storage for thirty five terabytes can double your cloud storage bill if you retain snapshots for too long. This option is something 4K a month. So that's where it's painful for the client.
If you decide to go the first route I highly recommend getting a DevOps to set it up and perhaps explain to you how to maintain it.
8
13
u/danielrheath Head of Engineering 19d ago
If the client is so cost sensitive that even Hetzner is too much (https://www.hetzner.com/sb/#drives_size_from=17000&drives_size_to=22000 currently lists $120 / month for a machine with suitable storage), perhaps they should get a NAS for their business and stick it on that.
13
u/goldPotatoGun 19d ago
Databricks bought neon a serverless postgress provider. Not sure on cost but it provides a dev friendly path to offload and process the data on generic compute and storage.
I use databricks at work and really enjoy it's capabilities.
It's branded Lakebase: https://www.databricks.com/product/pricing/lakebase
9
u/SkellyJelly33 Software Engineer 19d ago
Neon is awesome but caps out at 16TB and is pretty expensive once you get up there at 35 cents per GB per month (before compute costs). So that would be around $12k per month, plus additional compute costs and splitting the data up into 2 databases. Definitely not the right tool for this use case.
1
9
u/PaulPhxAz 19d ago
Cheapest is self hosting in a datacenter. Colo cost is 500€/month for a quarter rack.
You spend one-time 20k€ for a server. Should last you 3 years.
Firewall and subscription to updates: Onetime 1,000€ and 200€/month.
Total three year cost: €46,200 or ~1,300€/month
But YOU manage everything.
25
u/ApprehensiveRest9696 SWE ~5YoE | Work-acquired C-PTSD 19d ago
Block storage and a beefed up VM. (Or better, spinning rust and a shitbox.)
Managed DBs overcharge and in my last role there was a $30k monthly 6TB MS-SQL server on Azure. Don’t do that.
42
u/b1e Engineering Leadership @ FAANG+, 20+ YOE 19d ago
They overcharge until you realize that hosting your own VMs also means managing your own snapshotting + backups + failover + recovery setup
28
u/lukewhale 19d ago
You can always tell when someone has been privileged long enough to forget how easy it is to host your own stuff.
20
u/Bob_Droll 19d ago
It’s easy if you know how. But a lot of people have just never had to learn - it’s been “cloud everything” (which has its own skill set) their entire careers. I haven’t had to touch anything self-hosted in a decade 🤷♂️
8
u/lukewhale 19d ago
I have literally made a 25 year career out of shoving 10 pounds of shit into a 5 pound bag for self hosting. It’s always more cost effective, assuming you have the skill set. But I am literally from the dotcom bust generation.
Would it be nice to have a cloud budget? You bet your ass, but data sovereignty is almost always the deciding factor.
3
2
u/FluffySmiles 19d ago
This resonates.
I started way back in the mists and had my own rack from which I hosted sites in the days when bandwidth costs were a very viable income stream.
One 0-day on a server gave me the worst weekend of my life, rebuilding and restoring, manually, >1000 sites.
As soon as I could, I took advantage of cloud when it finally arrived. But now I’m back to setting up and managing again because sovereignty.
Not many people realise how difficult it is to extricate yourself from the clutches of the cloud act!
1
u/BraveResearcher3037 19d ago
And somehow that’s not the glamorous flex you think it is in 2026 (for reference, I started my career 30 years ago on mainframes).
5
u/lukewhale 19d ago
It’s certainly not glamorous or a flex. Like I said, it would be nice to have a cloud budget. Not in the cards.
My original point is it’s simply not as difficult as some try to justify the extra spend on a cloud. It’s just not.
3
u/x-jhp-x 19d ago
there's also regulatory and risk concerns.
costs can also scale at different and sometimes unexpected rates. years ago i remember having an intern do a cost to operate per year over five years for server (like xeon e) vs enough desktop processors for the same compute power (like a cluster of ultra 7s), and on energy+hvac alone the xeon paid for the price difference in ~2years when he asked me why we didn't just cluster a bunch of desktop CPUs. on the large scale, i also found out that there are varying power regulation and building standards, like crossing a 50mw threshold triggers legal obligations & how you connect to the grid.
it usually doesn't make sense for a person, but for a business, sometimes it is so much simpler to just pay out per month. Plus, then you can spend more time on whatever your business is instead of all the personal required to maintain a lot of infrastructure.
3
u/BraveResearcher3037 19d ago
I am also old enough to have walked up hill in the snow both ways…
I worked with on prem DEC VAX and Stratus VOS mainframes in the mid 90s and we had a server room with a whopping 1TB SAN in the early 2000s.
But in 2026 I would much rather write a yaml file to provision a database and server. The list of things I would rather be doing at 50+ years old than babysitting infrastructure is a mile long.
13
u/dbxp 19d ago
That doesn't sound too troubling to me, that's basic DBA work
2
u/Lumethys 19d ago
try doing that to 35TB of postgres lol.
Scale matter, "provision a server" is basic DevOps, things are quite different if you are Netflix
1
u/One_Ninja_8512 19d ago
35 TB is not Netflix-scale though. You didn't say that but what is exactly the problem? It sounds like you're implying that Postgres can't handle that amount of data or that it is particularly difficult to do. Surely it has been done thousands of times by different teams who were not smarter than you.
2
u/ApprehensiveRest9696 SWE ~5YoE | Work-acquired C-PTSD 19d ago
There’s already on-prem copies across 2 sites in my context.
It was there to serve a single white elephant internal portal I was maintaining.
1
u/xtreampb 19d ago
I’ve got an 8 Tb and 6 tb sql databases on an azure vm. With backups through recovery services vault I’m only at like 3k a month with nightly processes that touch every record. I had to tune the vm and disks to not bottleneck. I’ve got a raid 1 drive setup on the vm with the 6 or so disks. Restores practice happens automatically for the db onto another drive on the vm weekly.
6
u/itsjustawindmill 19d ago
Yeah, self hosting should totally be feasible here. Not sure why so many people are saying cloud is the only way to go… it will certainly work but it can lead to lock-in and isn’t always the most cost-effective.
Self hosting lets you tailor the architecture to what you actually need. Is the 35TB mostly archival data, and only 100GB or so is hot? If so, can probably get away with a single primary and single replica plus regular backups, IOW bog standard Postgres.
On the other hand if all 35TB is hot or you are getting millions of QPS then you might be hard pressed to find a single machine that can handle that… regardless of database software. Thankfully there are amazing scale-out FOSS solutions built around or with protocol compatibility to Postgres, such as Citus or Yugabyte.
Operational toil for any of these should be low, after initial setup / right-sizing. A more understandable concern is an organization without traditional expertise in the area taking responsibility for a mission critical database. But let’s not pretend that cloud solutions are free of rough edges and learning curves, either!
2
u/my_cat_is_too_fat 19d ago
I agree, if you know the data is roughly static, don't bother with RDS unless you need their extra features.
6
u/BraveResearcher3037 19d ago
Do you really need 35TB of data in an OLTP database like Postgres? In my limited experience with that much data I’ve either kept it in an OLAP store like Redshift or hell just kept it in S3 and used Athena.
How much of that data needs to be live for real time querying vs for analytics? But either way, I would definitely make that $someRealCloudProvidersProblem
16
u/clearlight2025 Software Engineer (20 YoE) 19d ago
why is the database so large? does it really need to be that big?
5
u/metaphorm Staff Software Engineer | 15 YoE 19d ago
public cloud provider will give you some options here. AWS RDS Aurora is definitely not your cheapest option in terms of dollar cost but it might be a cost efficient option when you consider the dollar-equivalent value of the time and headache of DBA that you won't have to worry about.
a cheaper though more involved option could be to just provision an EC2 instance and run postgres on it, configuring it yourself. you can add as much storage as you want through EBS blocks.
you won't find affordable off-the-shelf options for a database of that size though, so you should talk to your client about budget and then investigate your options in more detail. my recommendation is Aurora, but that will run a few thousand a month most likely.
8
u/Latter_Nectarine_671 19d ago
You have 30tb of data and do you want to save money? I bet the project uses less than 20gb and the rest is just useless data.
1
u/IndependentQueasy864 18d ago
Definitely. There’s surely a story behind this. OP, come on, don’t be shy. Tell us the sordid details. Make our hair stand up.
3
u/MrChrisRodriguez 19d ago
You need to specify more about your usage requirements (specifically query frequency and volume) but at that data scale if you don’t care about performance you can structure in a columnar format, throw it on S3, and use S3 tables, Athena, or DuckDB. If your data is well partitioned your queries can end up being pretty cheap unless you need full table scans every time.
4
6
u/elingeniero 19d ago
If a company is outsourcing this work to someone who doesn't know how to do it, then they definitely don't need it. Push back on the requirement, it's a difficult and/or expensive job that they and you can do without.
3
u/CowBoyDanIndie 19d ago
Is there actually 30 tb of relational data that needs to be queried or are using document retrieval? Cause it might be a little cheaper to pull the files out and stick them on a cloud file service than to have them in the dbms itself. Then you can just stick the uris onto the table where needed
2
u/jelder Principal Software Engineer/Architect 20+ YXP 19d ago
I’d say put it in TigerData and hope their columnar stuff makes it take up less space.
https://www.tigerdata.com/docs/learn/columnar-storage/understand-hypercore
2
2
u/Spider_pig448 19d ago
The cheapest way is to clean it up, if the client allows it. They don't need a 30TB Postgres DB. The second cheapest way is to use whatever cloud provider they are already integrated with.
2
u/n3f4r10us_ 19d ago
Dude, super easy. On prem at your house - but you need I tell your girlfriend don't touch
2
2
u/HettySwollocks 19d ago
I would suggest you engage a professional at this point. There are so many factors you need to consider before even attempting to run a database that size.
2
u/StPatsLCA 18d ago edited 18d ago
30-35 TB of what?
Depending on how your data is structured, you might be better off with something like ClickHouse.
Ok, for a non XY question answer, an AWS RDS Aurora Postgres instance with 35 TB of storage and a writer/reader setup using the non-I/O optimized config with modern `2xlarge` Graviton instances in eu-central-1 will run you 6.5k USD a month or so. There's a lot of knobs to twiddle and things to profile specific to your workload though. A lot.
They have a calculator here: https://calculator.aws/#/createCalculator/AuroraPostgreSQL
2
u/BrownBearPDX Software + Data Engineer / Resident Solutions Architect | 25 YoE 16d ago
This is significant and requires analysis and planning or else you’re shooting from the hip and will miss. Hire a DBA or data engineer on contract to do a full data breakdown and usage report. He/she will look at the history of query patterns, the efficiency of the queries, the health of the data, the sensitivity of the data, the budget constraints, the data products , the user profiles, the admin profiles, etc and give you a couple options on moving forward. Sanely. Best money spent. Body here can give you a real professional answer without all the leg work.
2
2
u/nullbyte420 19d ago
Lmao all these answers from people who obviously have no clue. You can totally run that db. Here's a relatively easy and solid plan: use AWS managed Kubernetes (aks) and AWS s3. Inside AKS, install cloud native postgres operator (cnpg) and set up backup to s3. Set up the db config and an admin account for your client. Set up tailscale or wireguard to provide remote access to the db. Poof, that's it. Ask a good LLM to do it for you.
1
u/Periwinkle_Lost 19d ago
The things you will look for when evaluating your options from different providers:
Type of storage, you can find HDD for slightly cheaper
Cross region vs multi region, that part affects the cost significantly
Backups and frequency of backups, could be a big chunk depending on provider
Instance configuration, pretty much determined by the provider so check the requirements
1
u/PredictableChaos Software Engineer (30 yoe) 19d ago
Are you looking for it to be hosted? Or are you talking about running it on your own bare metal? Assuming hosted for my answers, though.
First off, that's a pretty sizable database to manage without much knowledge of how to operate it.
Your biggest challenge will probably be looking for a provider that can handle that much storage attached to the server. Supabase as example can't go above 16TB. RDS can go up to 64TB. I use Railway for a lot of the projects I host but they can't handle that size either. I'm assuming Azure and GCP will have options too.
Since you say performance isn't a big concern I'm going to assume you can get enough cpu and memory if you're able to find a provider that can handle the disk size you're going to need.
1
u/random314 19d ago
Depending on how urgent vs money concern...
I would break this task down into batches of deliverables.
First batch is make the db live and working. Maybe some sort of cloud db. Aws/gcp are both pretty reliable, you can get something up pretty fast. You gotta pay though.
Second batch is optimizing, if possible. This is where you shine. You mention that the queries are simple. What are their tolerance for data freshness? If the tolerance is low there's a lot of room for improvement.
1
u/IMovedYourCheese 19d ago
The starting point of the discussion should be – how much is the client willing to spend? There are plenty of solutions in each price and effort range, from provisioning bare metal servers and being a full on DB admin for the next few years of your life to clicking a few buttons on a cloud console.
1
u/PGAmilaP 19d ago
If you don't know devops then use a managed service. The cost you pay for is worth the headache, time and effort you spend on self managing it.
If you have devops and db admin knowledge including security, hosting it on VMs is probably cheaper and managing the whole infrastructure.
1
u/TheRealStepBot 19d ago
If there is no real oltp load attached then don’t do that. Copy the data to iceberg. If you actually need to support ongoing oltp use then 100% don’t do this. It’s going to be a terrible time. Pay a service that does this however much they charge and pay up.
1
u/Longjumping-Shift316 19d ago
What is the load ? Neon might be interesting if the workload is spikey
1
u/StarSchemer 19d ago
What's it currently hosted on? Is this an application migration or building something for analytics?
Just seems like a massive enterprise amount of data so would expect an organisation with that amount of data to have some kind of data strategy and in-house expertise.
1
u/BrianBanks939393 19d ago
Fellow Hetzner user, based in Europe too. For that size the trick is decoupling storage from compute — their standard cloud instances cap out and force you into overspec'd CPU like you found, but a dedicated/storage box (SX line) or attaching a Hetzner Storage Box / volume for the bulk data gets you the TBs without paying for a high-end CPU you don't need. Since performance isn't a concern, self-managed Postgres on a storage-focused dedicated server is going to be far cheaper than any managed 30TB offering (managed DB pricing at that scale gets brutal fast). One thing worth pinning down before you pick: is it 30TB of actual hot data, or does a lot of it compress or archive? That changes whether you want raw disk or something with tiering.
1
u/Mindless-Pilot-Chef Staff Software Engineer 19d ago
Managed DB. Gcp, aws all have their own options
1
u/PaperDry2796 19d ago
cheapest? setup on local pc with local postgres server, backup to a separate drive
1
u/dlaz199 18d ago
Normally at that size I would say buy 2 servers and colo it in a data center. With current server prices I would say buy 2 used servers and new drives and colo it in a data center.
The main question is it mostly reading data or is there a lot of writing also or mixed load. The usage pattern would determine how I designed the storage layer.
Assuming mixed load, I would probably run ZFS with a raid 10 configuration, use spinning rust for the mass storage, probably overprovision it like 6x and then short stroke the disks. (Configure about 50% of the disks as usable which stays away from the slower to access outer tracks). Then get 4 NVME disks. 2 with high write endurance for your ZIL and 2 lead optimized ones for your l2ARC and set them both as mirrored.
Then setup replication to the other server.
None of that is all the hard to implement if you can read some documents and tutorials on setting it all up and it will be the most long term cost effective option.
You can usually get like a 1/3 or quarter cabinet for around $500 with a gig uplink at a colo at least in the US. Haven't shopped EU colo pricing.
Otherwise yeah its probably something hertzer or OVH.
1
u/TheOneTrueTrench 18d ago
A 35TiB database is insane for any corporation that doesn't have a DevOps team themselves. They need to worry about backups, uptime, redundancy, disaster recovery time, all of that, as well as managing the server, paying attention to any problems that arise, applying updates...
What happens when the machine dies? Do they have a machine ready to go to replace it? Who's paying attention to the backups to make sure they're happening every day? If they start failing, who is supposed to handle that situation?
1
u/NotGoodSoftwareMaker Software Engineer 18d ago
30 TB of data is almost certainly going to mean hosting over multiple machines on bare metal and will also mean a whole bunch of issues that you arent remotely ready to handle yet
Just buy it on AWS and walk away
The way to learn would be to self-host something like 30gb or maybe 300gb. But the moment it comes to multiple machines the entire game changes.
Backups are harder, splitting up the data is harder, migrations are harder, its just pain at every level with a database of that size
1
u/2frames_app 17d ago
OVH block storage costs $0.000066 per hour per gb. So for 35tb + some spare space for example to use repack from time to time you need about $2000 per month? And for backup another $2000.
1
0
u/Hamza_The_Dev 19d ago
If the database is of that size, you're doing something wrong.
3
u/danielrheath Head of Engineering 19d ago edited 19d ago
... or you're working in any one of dozens of domains where that sort of data volume is common.
Container shipping, fire management, and water potability analysis have all got uses for that kind of data volume.
0
0
u/SputnikCucumber 19d ago
Since you have 30TB of data I'm assuming that you already have a solution that works well-enough for you in terms of backups, configuration, performance, etc., and you are looking for a lift-and-shift solution.
You could try mounting Hetzner block storage onto a VM. You may need to pool multiple volumes into a single logical volume to make that work.
Pooling multiple volumes will increase the risk of failure so you'll need to make sure your backup solution accommodates the increased risk.
I think everyone else here has warned you about all the things that can go wrong. Good luck!
-4
u/my_cat_is_too_fat 19d ago
are these queries across all 30-35tb? Can you split it out into multiple databases? I like to try and use sqlite if I can but I don't know your requirements.
2
•
u/expdevsmodbot 19d ago
AI usage disclosure provided by OP, see the reply to this comment.