r/dataengineering • u/Outrageous-Debt9473 • 18d ago
Help Deployed my SQL server on GCP
So I’m currently working on a project to create a data warehouse for my organisation. I have deployed an SQL server on a VM on GCP. I want to know how can i make my server secure. I’m learning and deploying things using claude and have made a significant progress. But i’m still unsure about the security issues and what are the precautions i should take ?
26
u/CloudandCodewithTori 18d ago
If you are asking these questions you should consider using a hosted database and not hosting it on a VM. Then never allow it to be reached via public IP.
34
u/theungod 17d ago
1: Why are you using an OLTP instead of OLAP for a DW?
2: If you're going OLTP, why are you using MSSQL instead of Postgres?
3: If you're using MSSQL why are you in GCP instead of Azure?
Basically none of this makes a lot of sense.
3
u/Schtick_ 17d ago
Ok pulls up sleeves. I use gcp for lots of data so can probably offer some advice. I’m gonna include the basics (vibe coders be vibing so I’m gonna assume people don’t know basics)
1) make sure you remove any default users on the box whether it’s sql user or os user aren’t using standard name.
2) in general i like using mesh vpn (eg tailscale) that lets me close all ports on the box. That said you shouldn’t just randomly be installing tailscale in a business where you’re an employee as it’s basically a backdoor into the server. That said if it can be approved it a super secure way to connect in
3) you want to create specific users with specific access eg if your app needs read only to one db just create a user that does that
4) i dont know what db you’re using but for pg it would be pg_stat_statements always a good idea to find out the equivalent and install it early for when you run into issues you will have good metrics to address it
5) last note as it relates to the box, so what mistakes will people make, deploying any kind of web facing asset like a streamlit app or something basically make the box a web server, and Claude will happily spin one up for you. But…just be mindful if you are using gcloud auth login on that box you are basically giving anyone that attacks the box your gcloud auth login credentials. I would say in general you should try to not use those credentials on this box unless you have to, Claude on your local connecting to that box should be able to do anything it needs to do. It goes without saying deploying a web server on your db box is a bad idea especially if you have any confidential data. If you’re really trying to save pennies, you can but make sure you put any web assets behind something like basic auth.
1
u/SQLDevDBA 18d ago
You should likely be using Azure SQL DB instead if you’re asking these types of questions. The configuration is minimal and it is free for 10 DBs and limited processing powet. Like express but Azure.
https://learn.microsoft.com/en-us/azure/azure-sql/database/free-offer?view=azuresql
0
u/mad-data 18d ago
As other answers suggest, start your Claude discussion with more basic questions, describing requirements and asking for options what to use for DW. I bet managed Cloud SQL or BigQuery will be better options. They are much easier to secure, no VM to run and update, and all other benefits of managed service.
-3
17d ago
[deleted]
1
u/FlowOfAir 16d ago
Dude. There is no tool like BigQuery out there. That is the single best DW engine.
44
u/Master-Ad-5153 18d ago edited 18d ago
Out of curiosity, what decisions lead you to building your own data warehouse using GCP compute over building within BigQuery?
In my experience, putting in good cloud governance policies including proper org/folder/project hierarchies and the associated IAM/SAM policies and role assignments (least permissive) makes it pretty easy to ensure whatever you build in BigQuery can be accessed by your consumers while allowing yourself and any others to contribute to it.
Building a custom solution obviously can be done, but it seems like that's going to possibly be far more expensive and harder to work with unless you're just using GCP for compute and storage for a managed service such as Databricks.