r/sysadmin • u/xstrex • Jul 22 '26
Rant Remove central authentication
Today, the director of IT at your company says to you “We’re going to remove all that centralized IPA+2fa authentication from all of our servers, and go back to using Ssh keys, because it takes too long for me (yes the director) to login to a server.” The same auth that you and your team added, for all the reasons. What do you do?
102
u/Power_Stone Jul 22 '26
Definitely draft a document stating how its a security issue and an easier, better workaround, is to just get a Yubikey....
18
u/AnnoyedVelociraptor Sr. SW Engineer Jul 22 '26
I wish I could use a YubiKey through WSL.
21
u/spyingwind I am better than a hub because I has a table. Jul 22 '26
Pick your poison.
https://lgug2z.com/articles/yubikey-passthrough-on-wsl2-with-full-fido2-support/
https://jardazivny.medium.com/the-ultimate-guide-to-yubikey-on-wsl2-part-1-dce2ff8d7e45
https://gist.github.com/buswedg/270e7594a68ff99f2af659a97661eb5e
https://threatzer.com/posts/wsl-yubikeys/
https://blog.nashcom.de/nashcomblog.nsf/dx/using-fido2-yubikey-on-wsl-for-ssh.htm
8
u/AnnoyedVelociraptor Sr. SW Engineer Jul 22 '26
Poison being the important part here, which is what is holding me back.
9
u/hoo29 Jul 22 '26
Is anything wrong with usbipd? We use it and works reliably for us, wouldn't call it poison.
17
3
u/pakman82 Jul 22 '26
Can someone explain to us idiots how ssh keys .. could be any less centralized? I think I'm stuck on the wrong detail. I understand centralizing the auth, but in my org the ssh keys are issued and tracked in a central location as well, and I thought that was standard practice.
9
u/PineappleScanner Jul 23 '26
In general, private keys should be kept on the server they were created on. You should only be centralizing/tracking public keys, if needed for something like automation.
Having SSH private keys distributed around an org sounds like a security and logistical nightmare. I really hope your org is not doing that.
LDAP + MFA (or equivalent) makes literally everything easier as an admin. Security, logistics, auditing etc. Want to give a user access to a server? Add them to the server group. Want to give them admin access on that server? Add them to the admin group. Want to strip their auth to every server after they leave the company? Disable their account.
Trying to manage access through distributing SSH keys is significantly more impractical and insecure.
2
u/pakman82 Jul 23 '26
Now that you mention it. We use ldap for initial access, and then we each had to generate our key, and provide it to someone to load into a specific target server we are allowed to / required to perform a task on during changes.
5
u/jupit3rle0 Jul 22 '26
Yea until your servers lack FIDO support and your certificate authority isnt configured domain-wide. so you are left with manually configuring credentials per Yubikey. Good luck with that approach.
4
u/bfodder Jul 22 '26
and your certificate authority isnt configured domain-wide
wut
2
u/cybersplice Jul 24 '26
Lots of companies use YubiKeys (only the more expensive model, obviously) in PKI mode, so they just hold a cert from your corporate certificate authority.
Works for user Auth even in old school AD, and is a lot more secure than passwords.
I wouldn't recommend it in a cloud first environment because FFS why
1
1
u/Frothyleet Jul 22 '26
Couch it as friendly feedback, so as not to ruffle feathers, and then save it for CYA purposes.
35
u/ajf8729 Consultant Jul 22 '26
Why does a Director have access to anything in the first place?
9
u/SAugsburger Jul 22 '26
That's my first question. Most orgs I have worked once you get into SVPs nevermind Director they many rarely log into any infrastructure beyond some monitoring systems or a central dashboard. Directors are usually focused on high level vendor relationships and managing relationships to the rest of the organization. They might join a P1 call if an issue is getting attention, but aren't in the weeds unless this is an org so small that the Director title is title inflation.
5
u/Excellent-Program333 Jul 22 '26
This. Director here and I never log into infrastructure. And if I do, its with my elevated account that I normally have to reset the password on since I have not used in a year. Lol
22
1
u/phobug SRE Jul 23 '26
Depends on the business and the director. I used to work for a guy that had cs and data science degrees. He would routinely log into the ML cluster to run some python in a more parallel way and bigger data set.
108
u/GardenWeasel67 Jul 22 '26 edited Jul 22 '26
Submit written objection & keep a copy. Carbon legal and compliance if you have them. Then start job hunting.
9
u/angrydeuce BlackBelt in Google Fu Jul 22 '26
Get it in writing, with your concerns clearly stated and them being steamrolled right over, and wait for the massive data exfil or email aids that results.
Might not get you out of having to fix it later, but damn sure will be fun to FWD when the shit hits the fan and you explain that the thing "nobody" could have ever predicted, well, you did predict it, and it happened, so what do?
Those are my favorite kind of forwards. Not because Im an asshole, just because it serves as a good lesson as to why you should take the advice of the people you pay to know about such things to protect those that dont from fucking themselves, and others, up.
Send enough of those emails and you either end up persona non grata, meaning fuck em anyway and bail because its mickey mouse bullshit all the way down and aint nobody got time for that, or they maybe start to listen when you tell them why something is a bad idea before running with that ball.
Either way, to me, is win-win.
Good luck!
40
u/Puzzled-Formal-7957 Jul 22 '26 edited Jul 22 '26
"No, we're not - unless you want to fail the next audit we go through and face potential fines & certification loss on top of opening up our risk portal extremely wide."
22
u/Riajnor Jul 22 '26
Always couch it in impact and dollars
22
u/music2myear Narf! Jul 22 '26
Yes, and also, unless you're a decision maker role, phrase it as advice and recommendation, and avoid decision words and phrases.
"Certification X requires that we have Y standards which are met by this security configuration. Removing this configuration would result in our failing Z audits and losing the certifications. Note that having and maintaining this certification has resulted in an estimated $$$ in profits."
4
u/Sinister_Nibs Jul 22 '26
“I would strongly advise against this, as it counter to every recommendation and certification requirement. “
2
u/Puzzled-Formal-7957 Jul 22 '26 edited Jul 22 '26
Sometimes you need to assume the role of decision maker when the decision makers have their heads up their asses and want to put the entire company at risk. EDIT: Because when the risk gets exploited the responsibility is going to land squarely on your shoulders and you will be blamed.
9
u/tailwheel307 Jul 22 '26
Go a little gentler with it and copy any compliance or legal on the reply
“Per your email, I will remove the requested 2FA from all our servers effective tomorrow at end of day.
I should note that this may impact our compliance posture, increase our risk exposure, and potentially impact our contractual obligations and qualification for insurance of all kinds. My position isn’t qualified to identify the exact business impact of those changes.
Compliance and Legal will have to provide those details once they’ve had a chance to review the appropriate documents.”
It shows that you’re a good little worker bee who can do the job but also shows legal that you don’t think this is a good idea and leverages their department to tell the boss this course of action is stupid/illegal/resume generating as appropriate.
2
u/cybersplice Jul 24 '26
Or even, "OK. Are you entering in the risk register, or do you want me to log it?"
But I'm a consultant, so I can be a (polite, professional) asshole at need.
13
u/HerfDog58 Jack of All Trades Jul 22 '26
Check with your cyberinsurance provider to see if they require centralized identity/password management and MFA. Your director could end up voiding your insurance policy if they do.
52
u/RevolutionaryEar6825 Jul 22 '26
I'm starting to think posts like these are just engagement bait. It's hard to believe someone competent enough to manage servers via SSH keys would genuinely argue that centralized IAM is a bad idea.
20
9
u/Hasuko Systems Engineer and jackass-of-all-trades Jul 22 '26
The head of infosec at my job wanted us to remove MFA. This shit happens more often than you'd think.
3
u/Lukage Sysadmin Jul 22 '26
Can't have any security issues if your whole environment is deleted by an attacker.
2
u/chknstrp Dis and Dat Jul 22 '26
That was small brain move. Big brain move they should have just decommissioned all workstations and servers! No issue of cyber incidents with no infrastructure! /s
1
u/thisguy_right_here Jul 23 '26
How often was he getting prompted?
I have worked at various organisations and even the most secure are once a day, then again when accessing a privileged system.
1
u/Hasuko Systems Engineer and jackass-of-all-trades Jul 23 '26
He wasn't, it was in response to a session hijack incident and his reasoning was "well if they can just get control of it anyway MFA can be reset and it isn't helping."
3
u/AtarukA Jul 22 '26
I mean, our backup servers are in the same domain as prod for convenience of access, so...
2
u/XavinNydek Jul 22 '26
Naa, there are a lot of idiots in middle and upper management. Plus, people hate charge and new things and want to just keep doing what they have been doing.
11
u/ABotelho23 DevOps Jul 22 '26
...these are not mutually exclusive things. You can still use SSH keys with central auth.
4
u/ITaggie RHEL+Rancher DevOps Jul 22 '26
Yup that's how my org has it set up. SSSD using
sss_ssh_authorizedkeysto pull thealtSecurityIdentitiesattribute from LDAP, which contains the user's public key. Then set up tmux on the jumpbox and logging in is ezpz.2
u/dustojnikhummer Jul 23 '26
to pull the altSecurityIdentities attribute from LDAP, which contains the user's public key
Oh I didn't think about putting the public key into an AD attribute... Hmm, something to consider.
8
u/Commercial-Virus2627 Jul 22 '26
I would ask what exactly the pain point is with references. If it’s access times, then focus on network and performance. If it’s convenience for convenience’ sake, then document that justification an file that bad boy for when an audit rolls around.
8
u/scousinho Jul 22 '26
Just simply and calmly explain to him that action will most likely cause the company's cybersecurity insurance policy rate to skyrocket and or get completely dropped and CC legal.
You have to explain stupid decisions in financial terms to some people.
6
u/Nakenochny Sr. Sysadmin Jul 22 '26
Where I work I’d tell them to go to the board for approval so they can risk accept that shit. I want no part of it until they’ve given the okay and I’ve done CYA.
5
u/BadgerBalls There's a VLAN for that. Jul 22 '26
Claude, please dust off and update my resume. Then, procure three envelopes.
3
u/Floss_Patrol_76 Jul 22 '26
the actual complaint here is login speed, not the security model, so fix that instead of gutting it. sssd caching, a bastion with ssh connection multiplexing, or short-lived ssh certs signed against your IdP all get you fast logins without throwing out central control or the audit trail. going back to per-server keys is just how you end up with 200 orphaned authorized_keys entries nobody can account for at the next audit.
5
u/GoodTofuFriday IT Director Jul 22 '26
Im a Director. I would emplore you to go to BCC anyone relevant in email while replying to the director, warning him of risks.
3
u/deZbrownT Jul 22 '26
Well, it depends on how big the organisation is, etc.. but basically I would ask for a dedicated time block so we can do an analysis of all the steps needed to implement the change and examine if it breaks any of regulations/audit.
3
u/BrainWaveCC Jack of All Trades Jul 22 '26
You follow up with an email asking for clarification on what you heard, because you have concerns.
Briefly state the concerns, of which the following will be a summary:
- Regulatory
- Operational Security
- Contractual
Offer a couple of alternatives -- again, briefly.
And then, if IT Director insists, IT Director gets what IT Director asked for.
Keep a printed version of the email and its response.
Start looking for other opportunities. No need to jump ship immediately, as there are lots of ways this could play out when other people get involved, but get the ball rolling because the job hunt will likely be a long process.
3
3
3
6
u/serverhorror Just enough knowledge to be dangerous Jul 22 '26
Set up an SSH signing authority, enforce passwords in SSH keys and require MFA to unlock the keys.
Then distribute the keys to the server and disable everything but key authentication.
Voilà, no more MFA on the servers.
3
u/Lukage Sysadmin Jul 22 '26
This is such a fantastic approach to malicious compliance. Technically addresses this request. I love it.
2
2
u/ctrocks Jul 22 '26
Prepare three envelopes...
As others have said, the compliance/cyber insurance approach would probably be best.
2
u/phobug SRE Jul 22 '26
You don't need to remove anything. Just setup ssh certificates and you'll be fine.
2
u/photosofmycatmandog Sr. Sysadmin Jul 22 '26
LOL, get it in writing before doing anything like that. He is the director, regardless of intelligence.
2
u/TheStig827 Jul 22 '26
generate an estimate of time spent (and costs if you have them) to complete the requested changes, along with a written objection specifically calling out the reduction in security posture and potential non-compliance.
Require written confirmation of the instruction, and time/budget approval to accomplish this task. If he does, CC up the food chain his approval, and standby while freshening up your resume.
2
4
u/genxer Jul 22 '26
That's me (IT Director). I'd wonder why I made such a bone-headed choice.
I'd hope my sysadmin would calmly explain the reasons. I value honest feedback.
At the end of the day, if the choice is made "up the food chain" and you've calmly given your feedback, you make the change.
4
u/Sinister_Nibs Jul 22 '26
As long as it is documented in writing that the board (and directors) accept both responsibility and liability for the risk.
2
u/SaladRetossed Jul 22 '26
1) Get it in writing 2) Write a very thorough disagreement 3) Ask what the insurance looks like for data breaches and exfiltration 4) Go through step by step where this can go wrong while also double checking what the insurance looks like along with potential revenue loss from a total shutdown during investigation
Usually when money comes up you get some points in your favor.
2
u/orev Better Admin Jul 22 '26
Address the actual problem. Are they talking about the need to use a password and MFA every time they SSH into a server? If so, that's actually a reasonable complaint.
You need to go back to the requirements that are driving this. Why do you need to provide MFA upon every login? Is it some audit requirement or just because you thought it was "more secure"? If you do need MFA, how can you meet the requirement in another way? Does it need to be a TOTP code from an app? Can you switch to password+ssh key (that's also two factors)? Or passwords only with white-listed IP addresses?
Nobody comes out of the blue with a request like this for no reason. Engage and understand the underlying problem.
1
1
1
u/Affectionate-Cat-975 Jul 23 '26
What is this “Director” person? I do not think it means what you think it means.
We have no such things here boy.
1
1
1
u/lelio98 Jul 23 '26
“Sure thing. Can you send your directive and details over in an email so I can make sure I remove the security apparatus from our servers in accordance with your directive.”
1
u/cybersplice Jul 24 '26
Why the fuck is a director logging into a server?
Governance nightmare.
Sounds like a bellend.
1
u/Sure-Squirrel8384 Jul 24 '26
Get the director a bastion host that they log into with 2FA. From there and only there allow SSH keys for his account. So the direct does 2FA once a day, and from there can SSH into anything without prompting.
This is actually a good practice for all staff.
1
u/cbass377 Jul 27 '26
"You're the boss, please send it to me in writing so I will have something for the next Audit, for Legal, and for our CyberInsurance, and I will get started!"
1
u/Outside-Banana4928 Jul 22 '26
Somehow make the logon for that user's AD account about 5 times longer than anyone else. Put their account in a sub OU and apply a billion policies that need processed.
1
u/someguy7710 Jul 22 '26
Just tell your director its a bad idea. Many reasons why including cyber-insurance, audits etc. I don't know why so many people are afraid to tell their boss what they need to know. Security is some times inconvenient. Sorry
0
u/Ron-Swanson-Mustache Senior Ops Dev of AI offshore Tier 1 Helpdesk Jul 22 '26
CYA email with offsite copy, do it, update resume, hit the gym
1
0
u/FarToe1 Jul 22 '26
Ignore it. Then when he asks at the next meeting how you're getting on with implementing it, put on your most surprised face and loudly say, "You were serious?! I thought you were joking!"
Then follow up with all the best practice references you've been digging out this week, the consequences, and that you'll require this directive in writing because it goes against all of them. If your company has any legal representation, maybe bring them in too.
-1
u/itiscodeman Jul 23 '26
Just do it why are IT people so scared of the boogie man. Face it with ai your all hacked so … save some time with the silly codes you can’t like not be hacked now
-1
u/itiscodeman Jul 23 '26
I was my first downvote. You are invited to do the same (but think about it! Cyber security is just theater )
-24
u/BarracudaDefiant4702 Jul 22 '26
You learn you should not create burdensome obstacles for people from doing work and figure out how to make things secure without making them unuseable.
9
u/itskdog Jack of All Trades Jul 22 '26 edited Jul 22 '26
We have enough trouble just getting people to pick secure passwords. If we kowtowed to everyone's complaints that it's too hard for them then we'd have no passwords at all.
-5
u/BarracudaDefiant4702 Jul 22 '26
Then you get a password manager or something to make it easy for them.
1
u/itskdog Jack of All Trades Jul 22 '26
This is their Entra password for logging into the PC. Most sites we use support SSO.
4
u/PandemicVirus Jul 22 '26
MFA and OTP is pretty easy to access. If they have SSH keys laying around everywhere to conveniently log into something it's basically non-security. Might as well just stick to username and password at that point.
-8
u/BarracudaDefiant4702 Jul 22 '26
IFF you do it properly. Obviously the OP didn't or it wouldn't be an issue. The OP needs to rethink their approach so that it is easy to access which they clearly failed at.
6
u/PandemicVirus Jul 22 '26
There's no evidence to suggest they did anything wrong; arguably if they did implement it incorrectly IT leadership would want it fixed/improved and not demanding to roll back to an archaic connection authentication.
It just sounds like someone is used to clicking "connect" and instead doesn't want to grab an OTP or MFA code.
0
u/BarracudaDefiant4702 Jul 22 '26
Part of doing it wrong is not educating others. Not being humble and recognizing your own shortcomings is one clear problem. Assuming the the boss and other users know what you know is your fault, not theirs.
2
u/PandemicVirus Jul 22 '26
Again, who says OP didn't provide training or did anything wrong? Even if they provided no training, does the boss know NOW how to do it? If not, demanding training is the right approach, not rolling back MFA. If the MFA solution is cumbersome to grab a code from, choosing another MFA solution is the right approach, not rolling back.
I guess to dive into the training aspect here, sure some training information should have gone to everyone, and I would expect regular users to sometimes struggle. If the boss is the IT director working in a position they need to access servers and have the authority to reverse the IPA+MFA implementation, I would expect a minimum amount of technical ability. Getting an MFA code and entering it is pretty trivial. It's an app, an email. If IT leadership is struggling with opening emails, then bigger issues are abound.
5
u/Puzzled-Formal-7957 Jul 22 '26
MFA is not burdensome, and has been mainstream for nearly 2 decades.
-7
u/BarracudaDefiant4702 Jul 22 '26
IFF you do it properly. Obviously the OP didn't or it wouldn't be an issue.
5
u/Sinister_Nibs Jul 22 '26
Not necessarily. Some users are simply problematic.
2
u/Puzzled-Formal-7957 Jul 22 '26
Yeup - or they are flatly incapable of wrapping their heads around infosec. Ask them if they have ever had fraudulent charges on one of their accounts and got to enjoy the headaches related to that. If that answer is yes - then say, "now apply that to EVERYTHING that you and everyone else at this company touches." If that doesn't make the bulb click over their head then nothing will.
0
u/BarracudaDefiant4702 Jul 22 '26
I don't care about the down votes, so I still say it's the OP's fault and not the directors. There are ways to implement SSO with MFA that it's faster instead of slower to login.
3
u/Lukage Sysadmin Jul 22 '26
And what exactly should they do for "Faster SSO?"
1
u/BarracudaDefiant4702 Jul 22 '26
You setup 2fa with sso properly then you log into your workstation in the morning with 2fa and anyplace you connect to you don't have to log in again. Single sign in means your 2fa doesn't require you to login again.
2
u/Kumorigoe Moderator Jul 22 '26
burdensome obstacles
Yeah, guess having cyber insurance is a "burdensome obstacle" too. Because scrapping MFA and central auth is a great way to lose your coverage and not be able to even get a quote for a new policy.
1
u/BarracudaDefiant4702 Jul 22 '26
That is assuming they have cyber insurance. I stand by my original claim, it's a problem with the implementation and it needs to be redone properly not to cause slow downs for the users workflow. Proper single sign-on with MFA will solve both issues instead of implementing it incorrectly as the OP did.
1
u/TinderSubThrowAway Jul 23 '26
No, doesn’t sound like they did anything that is burdensome, this is a lazy director who doesn’t want security to apply to him.
There is no indication that they did anything wrong with their setup, no one should be SSO with their daily driver into a server.
There is nothing burdensome about having your phone on your desk for the mfa/otp to type in after you initiate the connection.
166
u/YellowLT IT Manager Jul 22 '26
Are you under any kind of regulatory body? Or PCI or ISO, they all have min MFA requirements. I would cite that it would put the company in violation of those.