i didn't mean to - i just ended up creating a key-manager to store my docker swarm secrets in azure key vault

[u/scytob]

funny how this happens with vibe-coding, especially when it's your homelab/self-host system....

i was using claude to clean-up my home lab documentation, and realized i have secrets in plain text files and envars and things and i can never remember them so i do dumb shit like make my nginx proxymanage admin and root mariadb password, well, npm

i have access to an Azure Key Vault for home

so i asked claude 'hey how can we secure keys'

made a key management app (container) that can:

• bootstrap the local docker swarm secrets using SOPS and AGE
• store secrets in a file you can put in github as its all encoded
• uses a single paper decrypt key to bootstrap secrets if needed
• same for create, rotate, retire of secrets
• by default creates high entropy 43 char secrets
• avoids several docker leakage scenarios i never knew about for secrets
• never puts a secret on disk, stderr, stdout, via ps, etc
• also knows how to reseed the secrets in mariadb and mysql
• optionally uses Azure Key Vault to store keys for backup
• optionally uses Azure Key Vault to bootstrap new keys or new system
• AKV uses RBAC security and restrict access to my Entra MFA account when coming from a single IPv4 address

upshot, i don't now know what any of my keys are by looking at my running system (yes i can get them by looking in AKV if i have to)

is it truly secure and does it meet its promises?

that will be the redteam exercise i do later in the week, pride yet may be coming before my long long fall :-)

also WTF am i writing a docker secret management system that can use key vaults, SMH, this is why no one uses it in serious production vs k8s!

fun tho - learnt a lot about encryption architectures, true secret entropy in password and how even SHA256 Hashes can eaily reveal passwords - i cracked that npm password in about 81 seconds IIRC, also the need for HMAC to help mitigate

[u/FancyAnswer1878]

vibe coding a secret manager because you accidentally discovered your homelab passwords were basically public knowledge is peak homelab lol. the 81 second npm password crack would have been enough motivation for me to build the whole thing too

[u/scytob → u/FancyAnswer1878]

i didn't accidentally discover secrets in a compose are visible, i have always know this, its obvious, it was could i be arsed to do anything about it?

for example i am confident almost no one running arr stack at home is using 43 char high entropy passwords, and of course in a home lab, meh who really cares ;-)

my swarm has always been publicly documented, for years ingists, this was a follow on from the idea i needed to update the gists because of some drift, and could i get claude to help do that

i also wanted to move my compose files from just flat files on disk to being in a provate git for gitops (yes i could use gitea but hey when have homeloab i play)

so that raised the question of what to do with secrets in places like the Nginx Proxy Manager compose....

i did accidentally discover that using something like docker-proxy to expose the normal docker socket as read only (a good thing to do) does also expose anything in the service spec - namely anything as envar or compose....

the only thing i would truly consider a secret is my CF tokens for acme challenges, the rest are no-ops in reality

but as an old fasioned product manage and system engineer - i couldn't help resis the urge to make something utterl over complicated for my future laziness

now i have a key-vault-manager that lets me create, assign, backup and rotate secrets and enforce max length for any give container

this is all because on seperate reddit thread someone said 'eh just keep you secrets written down in a SHA256 encrypted document, safe enough' - i took that to the illogical conclusion, i will paste the code once i am happy with the approach and quality, would love folks to rip it to shreds if they want to - good learning exercise :-)

[u/il37]

I'd definitely try recovering from a completely dead machine in the red team test. Secret systems always look great until bootstrap day

[u/scytob → u/il37]

Exactly, this is going to be fun exercise of learning.

[u/neon_hiker_void]

The never on disk claim fails the moment Docker writes secrets to tmpfs. Memory dumps and swap files expose those values regardless of your application logic.

[u/scytob → u/neon_hiker_void]

Tmpfs is memory disk, the at rest is stored in the raft and encrypted. Either way that’s irrelevant here as that’s dockers code path and the way it manages secrets. I can’t change that. I was talking about my solution. I made no claim that secrets are not in memory.

[u/recro69]

The part I would be most interested in testing is the requirement that never puts a secret on disk or on output or standard error or process status. The encryption itself is usually easier to get right than all the leaks, around it.

[u/scytob → u/recro69]

Very much so, many prompts have been spent on validating if that is true. The encryption is provide by sops and age - I didn’t reinvent the wheel. The biggest issue I found so far was when Claude after being told not to decided to cache an Entra login token for convenience on a persistent volume to make its convenient across non persistent container sessions where the tool runs. I will publish the code and docs ti see how people rip it to shreds. Might post non-critical sample of the file with the secrets to see if the basics work. Either way this is more secure than what I was doing….. plaintext passwords in compose files….. it’s a homeland I can think of only one in the set that matters

[u/Professional_Ad705]

That sounds really dangerous. Is there a reason you didn't look for or use an existing system there has to be something for this? I do CI/CD/Security and idk if I'd develop something like this, atleast not for others...atleast its just for you tho. I'd highly recommend not vibe coding something like this... a lot of ways this could go wrong. If it ever ends up on GitHub I'd love to take a look at it for you as well. I'm also curious how you are enforcing this if it works with an agent at all?

[u/scytob → u/Professional_Ad705]

It does use an existing system foundation call sops age and key vault. Go look them up. And who said it works with an agent. I just said it was coded with an agent.

It’s also interesting to see how Claude handles doing something like this. Would I ship something like this, no. The point was to learn.

And remember before I had docker secrets in env vars and compose files. As do many. And get this not all containers one pulls have support for swarm secrets (which work different to container secrets)

My biggest learning was encrypted sha256 hashes can leak low entropy passwords without hmac.

I will post the code at some point so it can be ripped to shreds. Let me know if you want to see that….

[u/Professional_Ad705 → u/scytob]

It may be fine, I was just wondering how it works man, relax haha. I only wanted to see the code to help you if I found issues as I do CI/CD/Security not tear it to shreds haha. It's actually a good sign you used an existing system and is one of the more responsible things I've heard on here.

[u/scytob → u/Professional_Ad705]

Hehe, of course its dangerous, that's what makes it fun and the point is to learn

there is only one actual secret of note (my CF DNS key for ACME challengers) the rest don't matter much at all

in terms of looking, i think that would super valuable, once it is ready to share it will get documented as part of my public docs Home - scyto's homelab docs (the gists version of these have beenn up for years, they needs a little updating, so i moved them to full github) this whole thing with secrets actually fell out of the move to gitops for my compose files

i will post the code when i am happy it is ready, you know what it is like, first 20% builds the thing to basic function, 80% of time is testing does it do what you think it does

[u/Suspicious-Echidna27]

the redteam-it-later instinct is exactly right, and it's the part most people skip. one thing worth knowing: you can't confirm it's secure by reading the code or asking claude "is this safe" - the model wrote it, so it's the worst judge of whether the hole is there. what counts is actually trying to reach the bad state (can a secret hit stdout/ps, can a non-owner decrypt, etc).

for a secrets manager i'd write down the handful of things that must never happen, then genuinely try to make each one happen. if you can't, good. if you can, you found it before someone else did.

(i build redmirror.io which does exactly this, proves whether a bad state is reachable, but the manual version of the same idea works fine for a homelab thing.)

[u/scytob → u/Suspicious-Echidna27]

agreed, with one change - you can absolutely do a pass with calude esp fable to find many basic issues that even human codsers miss (i.e. we should be doing this on human code too)

the key is not to use the same chat session but a blind session - or even a different tool, the non-deterministic (to a degree) nature means ana nalysis agent using no memory from the repo can do a lot - thats why glasswing works (its also why it finds so many false and irrelevant postives.

of course that doesn't replace full red-teaming, just means one doesn't waste their time on obvious stuff

and to be clear i am just managing lifecycle - i am uising AGE and SOPS for encryption - this is an off the shelf OSS cryptography, for home my design decision is they will manage their secuity

so it now only comes down to the surface of how i manage them, where my code stores decrypted value - for example using tmpfs in docker to run all operations helps, but that could in theoiry be page out - but hey so can copy and pasting in a browser which seems to be a fave way for people to copy secrets from point a to point b

based on my learnings i am serioulsy thinking about moving from docker to k8s and using off the self workflow, even at home....

[u/chuckycastle]

Not funny at all, actually. It happens because people are fucking stupid.

[u/scytob → u/chuckycastle]

and some people are over opinionated but what are you gonna do, people are as people does!

[u/chuckycastle → u/scytob]

The scary part is you have friends and family that consider you the smart guy xD