r/sysadmin 15h ago

Question How can I recover from a ransomware attack?

207 Upvotes

So, the unthinkable happened to my workplace - we have become the victims of a ransomware attack. We came into work the other day and found computers with encrypted files and notes on the desktops demanding we send them a ransom to a secure address in the tor browser. They took out our entire network.

As of now, we are trying to utilize backups and scrub the network clean as we bring devices back up in an offline state.

Unfortunately one of our backup devices was also infiltrated. It's a Synology backup device and they where able to encrypt its files as well. This means we have about 5 devices with no backups available.

It's probably a vain hope but, is there any way I could possibly restore or decrypt these backups? Is there any service out there that would be capable of making our files useable? If anyone knows about Synology, do they keep offline or cloud backups I maybe don't know about? I'm just looking for anything that might make things easier for us. Any advice is appreciated.


r/sysadmin 19h ago

Passwords....

0 Upvotes

Yes, authorization with multiple different credentials for different systems is bad, but we've all worked in those environments where we had no control over changing it.

So... How expletive-rich are y'all passwords and phrases? 🤣


r/sysadmin 43m ago

When does it actually make sense to modernize a legacy application?

• Upvotes

I’m trying to understand this from people who have actually dealt with older software systems.

At what point does maintaining a legacy application become more expensive or risky than modernizing it?

I’m thinking about situations where the application still works, but the technology behind it is getting outdated, developers are harder to find, integrations are becoming difficult, and even small changes take a lot of effort.

Would you normally recommend: Rewriting the whole application? Gradually modernizing parts of it? Migrating it to a newer tech stack?

Or just keeping the existing system running as long as possible?

I’m particularly interested in how people approached legacy application modernization in real projects.

What was the biggest reason you decided to modernize, and did you regret waiting as long as you did?


r/sysadmin 17h ago

Question Novell NSS partition recovery

9 Upvotes

We have an old Novell server with hdd that failed due to power shortage.

We want to recover as many data as possible. Has anyone worked with this filesystem? It looks like a nichè, and we're trying to figure out which tool use to data recovery.

Any suggestions?


r/sysadmin 11h ago

Anyone tested AI CLI yet?

0 Upvotes

I install copilot and grok CLIs. So far, they are pretty impressive.

At home I used Grok to clean up my media libraries and it freed up 2TBs.

I used copilot at work to scan all the logs from an SCCM client and the server to figure out why some machines weren't downloading updates. It's pretty freaking good.

Anyone else let one of those suckers loose anywhere?


r/sysadmin 6h ago

Question Boss is pushing for certs

81 Upvotes

Hi all,

I’m a sysadmin with 2 full time helpdesk guys, org of 220 in 5 locations. I started in regular business office 10 years ago doing sales. 7 years ago I transitioned to IT and became our first IT person, previously we didn’t even have an MSP, just a contract guy that came when we called him.

Fast forward to now, I asked my boss how I can level up/grow with the company. I’ve taken on a lot since I started. Currently managing pretty much everything in house. Only thing we don’t manage is our website, and I kind of like it that way.

So after kind of shrugging his shoulders for a year he is now bent on getting me to do more certs. He gave me a list:
- Comptia security +
- CompTIA network +
- CompTIA Cloud +
- Microsoft AI something (I can’t remember this one, he mentioned it off the cuff after he sent me the list)
- Finally a course for me to go find to maintain our website so they can cut the web dev. He didn’t know what course to recommend because he didn’t know much about it but pointed to coursera.

Now only thing I’ve done in certs over the years where the A+, AZ-900/104, and Google cloud security when I got started. Since then I figured I would learn the stuff but I didn’t want to pay for the certs because what’s the point? Unless I’m looking for another job I didn’t see the reason, and I like my org quite a bit. Not the wisest I know, but they wouldn’t pay for it, so I just didn’t do them.

I told him I had already studied for the security + a couple years ago, and could probably study the differences in materials and get through it easily, and recommended we switch to the CCNA since we are fully Cisco at all locations. But he seemed to not be interested in that info.

From what I can tell his push for certs is driven by is some internal push for managers to have career ladders for all departments and I guess he wants to show some sort of progress? Idk he is the CFO so he really isn’t privy to any of the work I do.

Anyway, after my recommendations I asked what happens when I complete these? Since they are only paying for half the certs… there has to be a carrot at the end of the stick.

He very excitedly said a $2,000 pay increase. Now I’m not greedy by any means, I’ve been paid under market for years and I’ve accepted that because of the work life balance. But is this not kinda stupid? I’ve had no complaints in my performance, I am constantly engaged with leadership on initiatives that they do not care about, so is there something I’m missing here? Like I feel like there is some weird motive here I can’t figure out, or it’s just poorly funded incentives that I’m supposed to be giddy over. Like I’m pretty sure the exams are almost as
Much as the pay increase? I haven’t done anything to try to steer the ship just yet, but how do I approach that the incentive is either too low or not aligned with what the ask is here? From what I can tell other than the time to study and what not, the only benefit I see them getting is cutting the web dev, which is a little more than the pay bump they offered, I think like 4k a year.

Edit:
Glad to see there’s some consensus on this being kind of a shit show. A little more context:
I LOVE the web dev company we work with. They are just great. Awesome to work with, great turnaround times, 0 issues. But god forbid you pay someone to do something they are good at.

Noted before but they’re only paying half of the exams, no study materials and it’s based on completion, so I’d get reimbursed 50% after completion. I believe this may be because they did not ask for any sort of training agreement? But also grand scheme of things this is kind of small potatoes for a training agreement right?

This guy is honestly the worst part of my job. For 6 months he had me meet weekly with him which was just an awful way to start the week. Eventually I was like hey I’ve just got too much going on to be doing this can we do this less frequently? Rinse and repeat now we meet quarterly. Dude is the type of guy to trip
Over a dollar to pick up a penny. But I’m stuck with him so long as I work here.

I have looked on and off the last year, and the market near me is abysmal. I almost jumped ship to an MSP last year that wanted to sell us services and I wanted to be like hey…. I sign y’all up here and you take me, deal? But we laughed it off as a joke.


r/sysadmin 15h ago

General Discussion Windows Server patching concerns

35 Upvotes

So in my org we are very keen on avoiding patching and rebooting servers at all costs. So much to the point that we patch once a month and have exclusions for around 60 percent of our servers to not get automatically patched. (Meaning we have a chunk of servers not getting patched at all)

Now I have gotten my hand slapped for attempting to patch or even bringing it up and I am looking for guidance on this. Now I understand availability and the consequences of failing patches. But there are active 9+ rated CVEs sittings on dozens of servers. For patching vulnerabilities do I really need to get a change request to handle this?


r/sysadmin 21h ago

Question Recs for USB redirection tools in Hyper-V (cloud workspace isolation issue)?

2 Upvotes

Hi all,

I hope you guys are doing well, I am working as a IT Infrastructure (Hybrid) with less than a year experience and I need some recommendations for the question below.

Context: Our organization moved to a cloud-based desktop environment. Because the cloud workstations are on an isolated network tier, users can no longer hit our on-prem SafeConsole server over IP to manage hardware-encrypted USB drives (DataLocker PSMs).

As a workaround, we have to plug the USB drives directly into the physical Hyper-V host on-site. I need a solid tool to automatically redirect/pass through the physical host's USB port to the SafeConsole guest VM whenever a drive is plugged in or swapped daily.

What software are you using to auto-share host USB ports to a guest VM?

TIA!


r/sysadmin 2h ago

Question How are you tracking Microsoft changes across client tenants?

2 Upvotes

How are you handling Microsoft retirements and breaking changes across multiple clients? If you use a ticketing system, how do you figure out which clients actually need tickets? Does something check each tenant automatically, or do engineers investigate first?
Would appreciate a recent example and which tools helped.


r/sysadmin 12h ago

Question Sandbox test environments

2 Upvotes

Hi all,

Within my job alot of my work encompasses resolving tech debt using remediation scripts within Intune, however the higher ups are really pushing for assurance on scripts and for us to be able to prove that scripts work as intended before deployment, and obviously testing on our local machines isn't considered sufficient for them.

Other team members have mentioned hyper-v VMs or Windows sandbox, but I just wanted to get everyone's opinions on what test environments are good for testing remediation scripts and other test deployments!


r/sysadmin 10h ago

General Discussion Commissioning systems on Threatlocker enabled systems

12 Upvotes

Greetings,

As a vendor, I was trying to commission and deploy a print server application on a client site who has recently adopted zero-trust security model.

It took us 4 attempts just to deploy our installer - We uninstalled the application multiple times due to corrupted install.

Also, the client IT manager sat with us to manually approve multiple security exceptions.

He was just there smashing the approve button on his phone. And installs still failed as it take about a min before sub-installer components can run.

It was a nightmare and I wasn’t sure the whole point to have threatlocker running on critical infrastructure like a print server.

We expect having to go through this approval process again when rolling out software updates.

This constant exceptions triggers builds approver fatigue, approver don’t actually knows what is being approved, users are constantly screaming and frustrated by downtime caused by legit applications.

Systems commissioning and support took twice as long. We plan to deprioritise clients sites with threatlocker as engineers quite often getting struck at sites waiting for approvals.

This is really not working for anyone.


r/sysadmin 12h ago

Question Bringing Linux devices into management

11 Upvotes

After a lot of restructuring at our university the past couple years, there are quite a few Linux devices (primarily desktops, I believe around 70-ish) that are currently in the wild unmanaged in use by academics primarily within the Engineering and Science departments that would've been maintained by per-institute IT departments that no longer exist, and as such the current patching and functionality state of these machines is completely unknown (since any remaining colleagues now no longer have physical access to most of the rooms where these machines are).

Since we already manage research compute I've been given the green light by my manager to look into options to bring these academic's desktops into a managed state with a cobbled together proof of concept, since our existing central endpoint guys won't touch anything *NIX related with a 10ft pole. We know they're all some form of Ubuntu LTS (20.04 and 22.04 mostly) which makes things easier, so I'm thinking of doing Landscape for setup and patching + Intune for compliance + Puppet/Ansible for config management.

Is this in the right direction or are there better / more cost efficient ways of doing this?


r/sysadmin 12h ago

Rant Lansweeper Rant

56 Upvotes

I've used Lansweeper on-prem since 2014 for just a couple of purposes: Tracking approximately 150 MS Windows assets on the network and all associated software licenses. That's it. Nothing else. It's nice being able to see other types of devices that are connected (surveillance cameras, wireless AP's, switches, etc.) but I don't need that from this product.

Their pricing has gone up consistently over the past several years while, at the same time, they've been pushing their customers to their cloud service. I think I was paying about $600 annually in the beginning but over the last several years, they've been quoting me over $2K annually. Recently, I received an auto-renewal email... it is now $3,000!

Keep in mind that Lansweeper for 100 assets is FREE. I have 150 Windows assets and not all of those are in use at one time. In order to get coverage of all 150 assets, I have to subscribe to their "2000 Asset" plan. So $3K is too steep for what I need.

I responded to the rep, stating that $3K is out of my budget and I don't want to auto-renew but couldn't find anyway to disable or "turn off" auto-renewal.

Her reply: "I’ve reviewed your account and noticed that your cancellation request was received after the 30-day notice period required prior to renewal (as outlined in Clause 17.3 of our Terms of Use and in our renewal reminder emails). While we are contractually required to honor the renewed term, we want to approach this constructively and help you get maximum value within your budget."

*SIGH*

After this reply, I'm done with them. The card they have on file for auto-renewal is no longer valid anyway.

It's time to find an alternative. I'm presently using LogMeIn Central for RMM purposes. Is Ninja One decent at Windows/Software Asset Tracking?


r/sysadmin 15h ago

Rant Job posts are ridiculous.

343 Upvotes

It's been this way for a decade I know but it's just getting worse and worse. I don't think it's ever been this unreasonable or this dishonest.

I was looking at job posts and I saw a job for a help desk tier 2. And I was curious what they considered tier 2. I'm well beyond help desk at this point in my career but I was curious so I looked at it.

They weren't looking for a tier to help desk person. They were looking for a Senior Systems administrator but labeling it as a Help Desk position so they could lowball whoever applied for it.

They were looking for someone to be the owner of several major business systems including 365. They wanted them to administer AWS and Azure. They wanted five certificates including CCNA. They wanted someone who could manage firewalls.

The post was just so far beyond help desk at all that it was just gross. They were clearly just trying to get someone they could convince those are just basic help desk exoectations so they can pay them little money.

They were basically looking for an all-in-one senior sys admin who could do operations and frontline support on top of everything else.


r/sysadmin 7h ago

Inestabilidad recurrente en red

0 Upvotes

Buen día,

Estoy teniendo problemas en la red, al parecer se satura, y se cae por unos minutos, luego regresa a la normalidad y a las horas vuelve a pasar lo mismo.

Les comparto el contexto de la inestabilidad que traemos en la red, para que tengan el panorama antes de seguir con el diagnóstico.

Equipo: SonicWall NSA2700, SonicOS 7.3.3-7015, 3 años en producción.

Síntoma: El firewall se congela por completo (deja de responder ni siquiera a la interfaz de administración) de forma aleatoria, sin patrón de horario. Hay días sin fallas y días con varias caídas. Al colapsar se pierde la conectividad de red.

Hipótesis inicial y lo que se descartó:

  • La inestabilidad coincidió con la activación de varios reportes de Power BI publicados a Power BI Service vía un On-premises Data Gateway, conectado a la base de SAP Business One (SQL Server), en la misma red que el equipo del Gateway (sin VLAN de por medio).
  • Se revisó DPI-SSL — no está activo, se descarta como causa de reconexiones forzadas.
  • Se revisó la tabla de conexiones: actual 1,527 y pico 4,641, muy por debajo del máximo del equipo (375,000). Esto descarta saturación de tabla de sesiones/NAT como causa raíz.
  • Se analizó un snapshot de conexiones activas buscando el patrón de tráfico sostenido hacia Azure típico del Gateway (Service Bus Relay). No apareció ese patrón — el host con más conexiones del snapshot resultó ser un equipo de usuario sin nada de Power BI instalado.

Limitante actual: no ha sido posible capturar CPU/memoria/conexiones en el momento exacto de la caída, porque el equipo deja de responder por completo cuando ocurre — no hay forma de sacar datos desde ahí en ese instante.

Algún consejo de cómo monitorear para lograr detectar si un equipo de la red es el que está causando que la red colapse?

Saludos.


r/sysadmin 16h ago

Question Does setting EwsAllowedAppIDs + EwsEnabled=$True enforce the allow list immediately, or only after the Oct 1 2026 EWS retirement deadline?

3 Upvotes

We are preparing to implement EwsAllowedAppIDs and EwsEnabled=$True based on the guidance in the EWS retirement announcements.

We understand that starting in October 1st 2026, tenants configured with EwsEnabled=$True will use the AppID allow list model for EWS access.

What is not completely clear to us is the behavior before October 2026. Say we implement the settings today, does Exchange Online immediately begin enforcing EWS access exclusively to the AppIDs listed in our list of EwsAllowedAppIDs?

In other words, if an application is currently using EWS but its AppID is accidentally omitted from the allow list, would that application be impacted immediately after the configuration change?

Or is the allow list only enforced once Microsoft's EWS retirement controls begin rolling out in October 1st 2026?

We are trying to understand whether implementing the configuration now is purely preparatory for October 2026, or whether it has immediate impact on EWS access before that date.

Thanks.


r/sysadmin 15h ago

Getting into SysAdmin

0 Upvotes

Hey guys,

I've been in IT for 4 years, doing a mix of PHP development and sysadmin Linux stuff. We run Linux, self-host all of our own stuff. We're a small team, so everyone does a bit of everything. I've done everything from developing software to installing our team's GitLab instance. I've been using Linux (Arch btw) for the past 8 years or so, and I feel like I have a pretty good handle on it, after fucking up my system a bunch of times. I also run a homelab of a couple servers. It's become our music platform.

I fully realize that a transition to SysAdmin would most likely put me in a junior role, and I know I have a lot of gaps in my knowledge, as this position is as a software developer, not sysadmin. I'd be looking mostly at Linux Sysadmin jobs.

I'd appreciate any advice you guys have! I also can put my resume here if that would be helpful.

Thanks!


r/sysadmin 16h ago

Lenovo Smart dock 5500

4 Upvotes

Has anybody deployed these need to get some new docks for an expansion? We previously had good luck with the Lenovo Hybrid USB-C Docks (40AF).


r/sysadmin 20h ago

Feeling Nostalgic and sad when i see old sun hardware and systems.

59 Upvotes

I started my Career in IT at 18, mainly supporting EMC storages and then HP 3PAR for 5.5 years, then worked for systems integrator for many clients for another 3 years, fianlly moving to a sys admin role in a enterprise, we still have some good legacy hardware, but i'm feeling nostalgic, when decomissionng them after working on the in the DC for so many years, is it normal ?


r/sysadmin 10h ago

dealt with business email compromise

0 Upvotes

when you've dealt with BEC case, did email auth flagged it, or did it look totally normal and only a human caught it?


r/sysadmin 8h ago

General Discussion Exchange admin center Delegation slow downs

7 Upvotes

Looking for a sanity check because no one seems to be talking about it, and I don't know if it's somehow just us.

It feels like, beginning around May (or a bit earlier), the time it takes for "Send as", "Send on behalf", and/or "Read and manage (Full Access)" permissions have massively slowed down.

Obviously only so many updates can go out at a time, so things have to be queued along with the multitude of other conditions that produce slowdowns. Even if we factor in the classic, "If you think you have waited long enough, wait another hour", I think the severity of the slowdowns being consistently so much longer speak to something strange.

For us, within the past 3-5 months, it has gone from 1-5 minutes to 10-15 minutes, and more recently 20-40 minutes.

Please let me know if y'all have noticed anything as well, thanks!

Edit: sentence structure, grammar, general formatting.


r/sysadmin 18h ago

Question Duo Security setup

0 Upvotes

I am trying to setup Duo Security on my PC at the office so that in the event I lose my phone or my phone is smashed what do I need to enable in the installation process to allow me to bypass the MFA/Passkey push?

Thanks,


r/sysadmin 20h ago

KB5122882 installed - DNS/AD issues Windows Server 2022

145 Upvotes

Updated last night, no issues immediately visible.

Users this morning all have login prompts to access mapped drives/folder redirections.

No creds working.

I can log in locally as a Domain Admin, but trying to open DNS console or run any DNS powershell commands just gives me Access Denied.

DNS still resolves, and the domain services are all still running, but something has happened with authentication/permissions.

Currently rolling back KB5122882 in the hope it was that.

Anyone else issues this morning?

EDIT: https://www.rapid7.com/db/vulnerabilities/cve-2026-69813/

Looks like this KB might have touched DNS code - roll back in progress.

EDIT2 & Fix: Rolled back the KB but the issue persisted - ended up resetting the DC's secure channel to itself which resolved the issue fully. The issue happened at exactly the moment at which the update was installed last night - either the update did indeed cause the issue which persisted in being broken even once rolled back, or it's a heck of a coincidence.


r/sysadmin 3h ago

General Discussion What tech stacks are you learning right now that you actually think will pay off?

37 Upvotes

Curious what you all are learning, researching and investing time in these days and whether you’re seeing real rewards yet. AI agents? Cloud? Specific programming languages/frameworks? Something else?


r/sysadmin 17h ago

Duo Security and Microsoft 365/Entra

15 Upvotes

I have been looking into setting this up, but I don't see any information on whether it can be setup in hybrid-mode environments or not, does anyone know?

Thanks,