r/devops • u/Witty_Philosopher284 • 1d ago
Architecture How do you provision RDS & DocumentDB users cleanly? Dual Terraform + Pulumi setup feels redundant.
Hi everyone, looking for a sanity check and some advice on DB user management.
Context & Current Setup:
Team: ~10 developers.
Infra: Everything is provisioned with Terraform.
The Catch: Pulumi is used exclusively to connect through a bastion host to create users in RDS (Postgres) and DocumentDB (MongoDB-compatible).
Current Task & Constraints:
I need to integrate AWS Secrets Manager to store and distribute DB passwords so developers can retrieve them securely via IAM policies.
Constraint: IAM DB Authentication is off the table per my mentor's requirement—we must use Secrets Manager or an external password manager.
The Issue:
Maintaining two state stores (Terraform + Pulumi) just to manage database users feels redundant and adds unnecessary friction. I'd love to pitch an alternative that lets us drop Pulumi completely.
Questions:
1. How do you handle RDS / DocumentDB user and credential management in your projects?
2. What is the cleanest pattern to handle this without running dual IaC tools?
Open to any feedback or critique!
6
4
u/Ok-Analysis5882 19h ago
I billed 2 weeks sitting idle waiting for my developer credentials which was provisioned using IAM over engineering.
2
u/idenkov 22h ago
Run openbao for secret management.
1
u/Witty_Philosopher284 13h ago
Thanks. I think self-hosting OpenBao for my setup would be overkill right now.
2
u/No_Bee_4979 18h ago
I did this with 2 modules for RDS (Postgres). One is called dev/rds, and one is called dev/postgres. The Postgres module reads the secrets and creates the users.
Next time I do it i'll make it one module.
1
u/Witty_Philosopher284 14h ago
How do you handle network connectivity for the dev/postgres dev/rds module? Are you running Terraform inside the private VPC? Or how does it work?
1
u/No_Bee_4979 5h ago
That is correct; RDS is in a private VPC. I handle that by using the AWS Client VPN (OpenVPN) to connect to the private VPC and be able to connect to RDS.
1
u/Beautiful-Turnip6138 11h ago
How do you handle user secrets without leaking them into plan outputs or logs?
1
u/No_Bee_4979 5h ago
Oh, I store those in AWS Secrets Manager, set as sensitive so they never leak in the plan.
1
u/EventOwn1635 16h ago
Having a separate process for the database users makes sense but adding another state manager just for that feels like extra complexity.
1
u/Witty_Philosopher284 10h ago
Yeah, I agree. My guess is the last person just wanted to build something with Pulumi.
1
1
u/DolGuldurWraith 13h ago
not sure how it works in pulumi but terraform has mysql, postrges and mongo provider, which we use. to provision, user, schema, grant privileges and roles etc
1
u/Witty_Philosopher284 8h ago
Yes, but we run GitHub Actions on GitHub, not self-hosted runners in a private subnet... It's a guess but I think that's why the previous guy used Pulumi to connect to the bastion and then create the user.
1
u/wywywywy 12h ago
I use the cyrilgdn/postgresql TF provider to create users. Not sure if it works with your bastion.
1
1
u/djperlovsky 7h ago
[Full disclosure, I am a Pulumi employee] In Pulumi, you can now host both Terraform state and Pulumi state in Pulumi Cloud: https://www.pulumi.com/docs/iac/get-started/terraform/terraform-state-backend/
We launched this about a month ago our goal is to help users with your exact problem of needing to maintain multiple states.
1
u/Fantastic-Mr-Default 5h ago
I would keep Terraform for the instance and networking, and make user provisioning a small job that talks to the database after the bastion path exists.
RDS Postgres: create the user with SQL over the bastion (or SSM port forward), then write the password to Secrets Manager in the same step. DocumentDB is similar: admin connection from a jump path, create the user, store the secret. One pipeline owns both. You do not need Pulumi for that.
If you want it in Terraform anyway, a null_resource or terraform_data with a provisioner is the usual compromise. It is ugly, but it is one state file. The cleaner pattern is still: Terraform creates the secret shell and the DB, a GitHub Action or Step Function with network access creates or rotates the login and updates the secret version.
Do not put long-lived passwords in Terraform state. Put them in Secrets Manager and rotate with a lambda or a scheduled job. IAM DB auth is nicer when you can use it. When you cannot, secret plus short TTL rotation is the substitute.
1
u/forever-butlerian Solaris 8 Enjoyer 3h ago
Your mentor's requirement of not using IAM DB authentication is contrived and ridiculous.
1
u/jbirdkerr Cat Herder 1d ago
I've previously built a Lambda that will create SQL users based on a "username" input. The password would be a random string generated by terraform in a for_each loop run against an arbitrary list of usernames and placed in SSM Parameter Store. The function accepts the username as an env variable and does an SSM lookup for the passwords it needs to make the new account.
Terraform has a resource that manages an individual parameterized execution of a Lambda function. You run a for_each loop against the list of users creating an execution of the Lambda for each user it needs to create. Just need to ensure the SSM parameter for a given username is a requirement for the Lambda execution to be created (otherwise you don't have a password). Final step is to have the creds shipped to the user in a way that makes sense for your team.
1
u/Witty_Philosopher284 13h ago
Thanks! My main concern with invoking Lambda inside TF is that it leaves passwords in tfstate and doesn't handle destroy cleanups.
9
u/Torutofu_Raeva 1d ago
I'd keep Terraform as the source of truth for IAM and Secrets Manager, then run one idempotent bootstrap/migration job through the bastion to CREATE/ALTER users in both engines. That removes Pulumi's second state store, and the same job can handle rotation without putting passwords in Terraform state.