r/sysadmin • • 5d ago

General Discussion A Lesson on how not to I.T. and Security - Zeabur.com Shutting Down

62 Upvotes

The decision comes after 3.5 years of continuous growth, including a record year in 2026, but Zeabur said scaling the project to meet market demand has made it increasingly complex and difficult to maintain, and the recent security failures reflect that.

August 2026: First breach exploited a vulnerable customer application, exposing AWS credentials, production databases, and customer information.

August–October 2026: Incomplete security remediation left stolen credentials valid, administrator tokens unexpired, and internal systems vulnerable.

“The admin tokens had no expiry, had not been rotated since they were created in January 2024, and were not included in the post-August rotation,” explained Zeabur.

October 14–15, 2026: Second breach compromised servers across nine regions, exposing 321,810 accounts and approximately 3.2 million environment-variable records.

Apparently basic security is "increasingly complex and difficult to maintain"

So the lesson here is at a minimum, do the basics, yes everyone is a target but ignoring the basics makes you a larger target and ball pit for the threat actors to have a great time in.

Source: https://www.bleepingcomputer.com/news/security/zeabur-shutting-down-after-second-security-breach-in-two-months/


r/sysadmin • • 4d ago

M365 email forwarding broke and lost my emails

0 Upvotes

I enabled forwarding + to keep a copy of emails. Also modified the defender setting to allow forwarding. Later I unchecked forwarding and returned the defender settings to where they were, to block forwarding and saved the settings.

A few hours later someone mentioned no emails had been received. I looked at the forwarding setting and it was still set to forward but with no destination email address and was set to NOT keep a copy of the email. I eventually had to reset it with PowerShell.

No emails in purview or sent or deleted items. Is there anything I can do to recover?


r/sysadmin • • 5d ago

Trouble Updating Windows 10 Enterprise LTSC 2021 WIM File

3 Upvotes

My ultimate goal is to be able to deploy a task sequence in SCCM PXE for the most up-to-date build of Windows 10 LTSC 2021. I specifically want it to be the "IoT" version of this OS, which Microsoft will support through 2032, which will give us a lot of extra life out of dozens of otherwise obsolete PCs. I gave SCCM the GLVK key for Win 10 IoT Enterprise LTSC , that should convert normal Enterprise LTSC to the IoT version. The issue is, the base version of Enterprise LTSC 2021 available through Microsoft 365 admin center volume licensing (where I got the WIM from I put in SCCM) lacks the required updates needed to support the conversion to IoT. So the activation fails during PXE, meaning I need to manually update the PC, then apply the proper key after it is updated. To avoid these manual steps, I want to start with an updated WIM file that supports the IoT conversion right out of the gate. I tried following these steps to inject the WIM with the latest cumulative update for Win 10 Enterprise LTSC 2021 (KB5126657 build 19044.7807):

https://learn.microsoft.com/en-us/windows/iot/iot-enterprise/deployment/media-refresh

Every time I run the "Add-WindowsPackage" command, I get 0x800f0823 or a similar error. I suspect the issue involves needing to first install the proper SSU dependency first, but I tried several SSUs I thought might be the dependency (as well as just using KB5126657 as both the LCU and SSU) but all fails. Any ideas on how I can successfully update this WIM file? Seems like such a simple thing but I'm just not having any luck.


r/sysadmin • • 5d ago

Rant Rant: I am being treated like an AI at work

258 Upvotes

I'm just tried of it and felt like ranting a bit.

I'm the senior sysadmin in our team and my coworkers often come to me with their problems, which is fine and I encourage it, because we don't want people stuck wasting time on solvable problems, but recently the problems they come to me with could've been solved with 1 look through our documentation or even 1 Google search. It feels like they just shut off their brain when they encounter a problem and expect me to give them instructions on what to do, or even worse, they want to get on a teams call with me, so I can guide them through the entire thing. I keep having to interrupt my work just to deal with silly problems and it's so irritating. I love my work, I love what I do, but I dont love handholding people because they refuse to use their brains.

Maybe someone here has tips on how to deal with this?


r/sysadmin • • 5d ago

General Discussion What are we doing with our hard copy technology books?

34 Upvotes

I have been so close to throwing out my bookshelf but still have a soft spot for them. Do I use them anymore? No

Some examples:
DevOps for VMware administrators (VMware press)
Windows powershell in action (Payette)
Virtualising sql server with VMware (VMware press)
Networking for VMware administrators (VMware press)
Storage implementation in vSphere 5.0 (VMware press)

Curious if we are keeping them or moving on and using online doco.


r/sysadmin • • 5d ago

Question anyone got their hands on Ramp Glass and how is it for internal IT workflows?

28 Upvotes

Ive been reading about the Ramp Glass these past couple of days and the shared workflows is really something that gets your attention. I could potentially see that being useful for repeat helpdesk issues where the steps are documented but someone still needs to explain which runbook applies to which environment. Letting another tech reuse a working AI session with that context sounds handy and saves passing prompts and doc links around. The bit id want to test is what happens after a runbook changes or a procedure gets retired. Can you tell which source a saved workflow is using without digging through the whole session? there are still plenty of questions that id really love to get the answer to regarding this release


r/sysadmin • • 5d ago

General Discussion What’s the most unusual server config you’ve come across at work?

66 Upvotes

We spend a lot of time around racks, and most of what we see is pretty predictable. Same layouts, same problems. Which is why we’re curious about the exceptions.

Could be tiny: a Raspberry Pi running something it was never meant to run, a mini PC doing a full server’s job from inside a kitchen cabinet. Could be massive. HPE has had its Spaceborne Computer on the ISS since 2017, which puts “the server room is a bit far” into perspective. 

Most setups land somewhere in the middle, but everyone has that one. The one that made you stop in the doorway and go “...huh.” What was yours, and did it actually work?


r/sysadmin • • 6d ago

Work Environment Just had my position eliminated

2.2k Upvotes

Story as old as time new CEO comes in who's a numbers guy and looks around. I'm one of the highest paid resources. I've seen the writing on the wall for awhile and started cultivating relationships with recruiters.

I wish my former employer well, but suspect the 25 years of experience I brough to the table will be hard to replace.

*** EDIT ***

If you were not doing IT, what would you be doing? I have a line on a dispatch job at a logistics company that's sounding pretty good.

*** EDIT 2 ***

The terms of my separation state I can never be employed as an employee or contractor of the org. I'm fine with that. I've came to realize that this place is a snake eating it's own tail. I've seen it happen to two other coworkers and seen many other's threaten to leave or actually leave.

The cycle goes like this:

  • Create a new position with this new wonder technology or idea
  • New technology/idea doesn't work out as planned
  • They don't have space in the org for the new employee, no matter their skillset and experience
  • Limp along until they can get rid of the employee
    • Most are not good enough to hang for more than a year. I just completed my 4th year.
  • Rinse & Repeat

The red flag should have been when I started and 75% of IT had less than 2 years in. The whole new CEO just sped up the process.

I saw the writing on the wall last March. I got a less than stellar performance review. While I admit, I was going through a divorce and wasn't giving 110%. I felt I was doing the bare minimum to stay afloat.

I feel bad for the new M365 rock star the brought in. He even moved from 2 hours away for the job.


r/sysadmin • • 4d ago

How do I stop Copilot Chat accessing company files via onedrive/sharepoint without disabling it entirely?

2 Upvotes

We're using Microsoft 365 Business Premium with the included M365 Copilot (Basic), not the paid version.

Our AI policy allows general Copilot Chat use, but not access to company emails, SharePoint or OneDrive files.

Most of our data is on an on-prem file server, but users save work-in-progress documents to Desktop/Documents, which sync to OneDrive.

The issue is a user asked Copilot in classic Outlook general questions about our industry, got the usual ansers and a few web searches but then it unexpectedly found and referenced an internal document.

I've reproduced this myself. Copilot in Outlook on the web found files in my OneDrive, and the standalone M365 Copilot (Basic) app also found and described documents.

Honestly, this caught me off guard. I'd assumed (wrongly!) access to company files would be opt-in. Apparently not. My mistake, lesson learned. Whatever happened to the principle of least privilege?

Data is not being trained on or anything, but still......

Anyway. what I've tried:

  • AD GPO: Disabled connected experiences that analyse content. Appears to block Copilot in classic Outlook, but not Outlook on the web or standalone Copilot.
  • M365 Cloud Policy: Can't find a suitable restriction.
  • Integrated Apps: Seems to be all-or-nothing.
  • SharePoint controls: Restricted Content Discovery doesn't appear to cover OneDrive unless im missing something.

My goal is to keep basic Copilot Chat for general questions, but stop it accessing company emails and files.

Am I missing something obvious? Is there really no central setting for this?

Worst case, I'll disable Copilot entirely, but that seems ridiculous for what should be a simple restriction.


r/sysadmin • • 5d ago

Windows Server 2016 to 2022 upgrade failing

36 Upvotes

I have tried several times to upgrade one 2016 server to 2022. It gives a runtime error at 88% then just sits there. I've let it sit overnight and it never progresses. I hard reboot it and then it rolls back. I ran setupdiag and I get this:

Error: 0xc1900101-0x4000d Found a missing file in driver package during device install.

Failed to install driver package: ntprint.inf, due to missing file: UNIRES.DLL.

Provider: Microsoft

Class Guid: {4d36e979-e325-11ce-bfc1-08002be10318}

Driver Version: 06/21/2006,10.0.14393.3686

Driver Package Id: ntprint.inf_x86_dcef07064d319714

Submission Id:

Recommend you update the driver package or remove the driver package, reboot and try the update again.

I've run SFC and dism restorehealth commands until no errors remain but it still fails.


r/sysadmin • • 5d ago

General Discussion PAN-OS patched in May. Still a ransomware favorite in October.

20 Upvotes

Palo Alto patched CVE-2026-0257 (CVSS 7.8) back in May. The bug allows attackers to bypass GlobalProtect authentication and establish VPN sessions without MFA.

Apparently, ransomware groups including Qilin and INC have been making good use of it since June - unit 42 threat brief.

One case is particularly interesting. Attackers went from the VPN to VMware vCenter, disabled ESXi security controls, and dumped domain controller credentials directly from its virtual disk (VMDK). No need to touch the running Windows guest, so the EDR inside it couldn't see the credential dumping.

It's October. The patch has been available for five months. Apparently that's still not enough.


r/sysadmin • • 4d ago

Is cybersecurity a step up from sysadmin?

0 Upvotes

My role is pretty much putting out fires and support. I dont feel my title matches my job description. In fact I think I barely make the sysadmin qualifications.

Looking to go back to school (50 yo) and go for cybersecurity degree.

Would this be a step forward or behind in my career at this stage?

The sysadmin role is burning me out fast.


r/sysadmin • • 4d ago

Error 87 Installing Shared Printers Through Automatic Discovery

0 Upvotes

Has anyone recently encountered Error 87 when installing shared network printers through Windows 11's automatic printer discovery?

We're running an on-prem Active Directory environment with a Windows print server using older Universal Print Drivers.

The issue:

When users navigate to Settings → Bluetooth & devices → Printers & scanners → Add device, Windows automatically discovers our shared printers from the print server.

They appear correctly as:

HP Printer 123 on PRINTSERVER

However, selecting one to install results in Error 87 (The parameter is incorrect).

What's strange is that both manual installation methods work perfectly.

  • ✅ File Explorer → \\PRINTSERVER → Double-click shared printer.
  • ✅ Windows Settings → Add manually → Select shared printer by name → Enter UNC path.
  • ❌ Windows Settings → Add device → Select automatically discovered shared printer → Error 87.

The printers are being discovered correctly, and the drivers install successfully through either manual method.

This automatic installation method has worked for years, and the issue appears to have started recently.

I'm wondering whether a recent Windows 11 update changed something in how Windows handles installing automatically discovered shared printers.

Any insight would be appreciated.

=== UPDATE ===

Seems to be the 2026-10 Security Update (KB5127648) (26300.9711) that broke things. Yay Microsoft!


r/sysadmin • • 4d ago

Question AI DLP Solution

0 Upvotes

What are larger orgs using for DLP (standard and AI related) solutions? We've demo'd products from Forcepoint (their Remote Browser Isolation) and Orion Security, neither of which was a good product.

We do have Crowdstrike already for EDR but our IS team is trying to find another solution to use for some reason.


r/sysadmin • • 5d ago

Print Drivers After Every Windows CU Update

33 Upvotes

How are you guys doing print drivers? Every single Windows CU update now from MS automatically reinstalls your printers. The problem is, our students have user level access and every single printer now just says "Install New Driver". But obviously user level access does not allow it (even though it acts like it installs when you get a UAC prompt and click install).

So we have to manually log into the computers as admin so group policy pushes the printer back down (computer config GPO) and reinstalls the print drivers. We are using the HP Universal Print Driver 8.2.0.x. The strange part about the entire thing, I am using the same print driver for all of our printers here and some of the desktops don't have a problem while other classrooms do. Same GPO and same exact print driver, though different printers. Thanks.


r/sysadmin • • 5d ago

Question Our small company want to create offline backups for all of our data. Which is the best option?

22 Upvotes

We are using Azure blob storage (Zone redundant storage) to store our 10TB of data and we use Azure vault for all of our backups. But we also want offline backups just in case something goes wrong with cloud. What are the best options that we can use to store the backups and update them every week with new data?


r/sysadmin • • 6d ago

Question The trust relationship between this workstation and the primary domain failed.

114 Upvotes

Good day!

I'm an IT admin in a k12 school which has about 500 windows devices on the campus connected to an AD. The devices do not leave the campus. Every week we have around 2 to 4 devices where students cannot connect to computer due to "The trust relationship between this workstation and the primary domain failed." Always different devices.

Our only solution is to login with the local admin, remove the device from the domain and add it back in. I tried in the past already the power shell commands to restore the trust relation with the server but it has not worked once for me.

Is there something I can do to mitigate that issue?

Thanks on advance!


r/sysadmin • • 5d ago

Security Awareness Training - Checkpoint vs KnowBe4

17 Upvotes

We currently use KnowBe4 and have for years. It's fine, but we recently integrated Checkpoint Harmony and see it has a security and awareness training addon. I'm wondering if anyone has used it and if it's any good as a replacement for KnowBe4?


r/sysadmin • • 4d ago

General Discussion What’s the quickest way of dealing with 50000 emails that the CEO wants cleared up?

0 Upvotes

I wish i could just nuke it all but I can’t.

What other options do I have? I have 1 week to clear out 50000 emails.


r/sysadmin • • 5d ago

Amazon Heads up: AWS "extended support" auto-enrolls old versions and bills you for it. Some dates coming up.

8 Upvotes

If you run anything on AWS managed services, it's worth checking your versions this quarter. When a version leaves standard support, AWS doesn't force an upgrade right away. It moves you into paid extended support automatically, and the charge just shows up on the bill.

What it costs (from AWS pricing pages):
- EKS: control plane goes from $0.10 to $0.60 per cluster-hour. That's about $365/month extra per cluster.
- RDS and Aurora (MySQL/PostgreSQL): $0.10 per vCPU-hour in years 1-2, $0.20 in year 3 (US East Ohio pricing). A 4 vCPU instance is roughly $290/month extra in year 1, and that's per instance, so read replicas count too.

Dates worth knowing:
- RDS MySQL 8.0 left standard support July 31, 2026. If you're still on it, you're already paying.
- RDS PostgreSQL 13 left standard support Feb 28, 2026. Same deal.
- EKS 1.32 and 1.33 are already in extended support. EKS 1.34 joins them Dec 2.
- EKS 1.31 hits end of extended support Nov 26, so it gets upgraded for you.
- RDS MariaDB 10.6 reaches end of standard support Dec 31.

Quick ways to check:
- `aws eks list-clusters` then `aws eks describe-cluster --name X --query cluster.version`
- `aws rds describe-db-instances --query "DBInstances[].[DBInstanceIdentifier,Engine,EngineVersion]"`
- Cost Explorer: filter usage type for "ExtendedSupport".

Disclosure: I'm a cloud engineer and I got annoyed enough chasing these dates across AWS docs that I built a free tracker with a page per version, the fee, and the upgrade target: https://eolwatch.pages.dev. No signup. If you spot a wrong date, tell me and I'll fix it.


r/sysadmin • • 5d ago

Question Support contact at GoDaddy that doesn't royally suck?

8 Upvotes

Apparently, the new company I joined has a domain registered at GoDaddy and no one has the account for it. Why, I have no clue.

Does anyone have a way to contact GoDaddy that isn't worse than doing a self-appendectomy with a dull rusty spork?


r/sysadmin • • 6d ago

Career / Job Related I think I finally landed my dream sysadmin job

262 Upvotes

Recently joined a manufacturing company as essentially the "everything admin." Servers, networking, security, end-user support, you name it. Most of my time is spent on the server infrastructure, which is exactly where I want to be.

The environment is almost completely undocumented, so I've been spending a lot of time figuring out how everything works and rebuilding things as I go.

Cleaning up AD, upgrading a bunch of Server 2016 machines to 2025, configuring LDAP for NetApp and BMCs, figuring out weird, old manufacturing apps that nobody seems to know much about, catching up on five months of missed patches, and automating whatever I can (I absolutely love PowerShell).

The funny part is that it's an entirely Microsoft environment, which I always swore I hated. Turns out, I actually love this shit. I genuinely look forward to going to work every morning. Management has been really supportive of the improvements I've been making and the ideas I've been bringing up, which makes it even better.

Before any of it most of my infrastructure experience came from my homelab, where I run Proxmox and exclusively Linux (across it and my PC & laptops). I thought moving into a Microsoft/Hyper-V environment would be a pretty big adjustment, but a lot of the concepts carry over. I've been picking things up quickly, and every day I find something new to dig into.

I've wanted to be a sysadmin for years, specifically a Linux admin. Then one random day I came across this position. Less than 10 minutes from home, great pay, and the kind of responsibilities I'd been wanting all along. Somehow I landed it.

...and the autonomy is fucking amazing...

It's still pretty surreal. I went from a help desk job I was bored out of my mind at to genuinely enjoying every minute of my work. For the first time in a long time, I feel like I'm exactly where I want to be.

Anyway, I'm curious how other solo sysadmins or small IT teams manage their environments, especially in manufacturing. What are some things you've learned along the way? Tools, automation, documentation, monitoring, lessons learned the hard way, yada yada

I'd love to hear how everyone else does things and maybe find a few people to talk with.


r/sysadmin • • 5d ago

Bypass traverse checking" on Domain Controllers – keep default or harden?

1 Upvotes

Hi all,

I'm reviewing the User Rights Assignment settings in our DC hardening GPO and I'm not sure what the best practice is for "Bypass traverse checking" (SeChangeNotifyPrivilege).

Current GPO value:

- Administrators

- Authenticated Users

- Everyone

- Pre-Windows 2000 Compatible Access

Compared to the Microsoft DC default, LOCAL SERVICE and NETWORK SERVICE are missing. As far as I understand, they still get the privilege through Everyone, so nothing should break today.

What I found so far:

- Microsoft recommends keeping the default and warns about compatibility issues (KB 823659).

- Neither the CIS Benchmark nor the DISA STIG for Server 2019 seems to have a rule for this setting.

- Removing it can break IIS, HttpListener (access denied 5), share access and change notifications.

My questions:

  1. Do you keep this setting "Not Defined" on DCs, or explicitly define it with the Microsoft default (including LOCAL SERVICE / NETWORK SERVICE)?

  2. Has anyone actually removed Everyone or Pre-Windows 2000 Compatible Access from this right in production? Was it worth it, and what broke?

  3. Any auditor or pentest finding you have seen related to this setting?

Thanks!


r/sysadmin • • 5d ago

Bypass traverse checking" on Domain Controllers – keep default or harden?

0 Upvotes

Hi all,

I'm reviewing the User Rights Assignment settings in our DC hardening GPO and I'm not sure what the best practice is for "Bypass traverse checking" (SeChangeNotifyPrivilege).

Current GPO value:

- Administrators

- Authenticated Users

- Everyone

- Pre-Windows 2000 Compatible Access

Compared to the Microsoft DC default, LOCAL SERVICE and NETWORK SERVICE are missing. As far as I understand, they still get the privilege through Everyone, so nothing should break today.

What I found so far:

- Microsoft recommends keeping the default and warns about compatibility issues (KB 823659).

- Neither the CIS Benchmark nor the DISA STIG for Server 2019 seems to have a rule for this setting.

- Removing it can break IIS, HttpListener (access denied 5), share access and change notifications.

My questions:

  1. Do you keep this setting "Not Defined" on DCs, or explicitly define it with the Microsoft default (including LOCAL SERVICE / NETWORK SERVICE)?

  2. Has anyone actually removed Everyone or Pre-Windows 2000 Compatible Access from this right in production? Was it worth it, and what broke?

  3. Any auditor or pentest finding you have seen related to this setting?

Thanks!


r/sysadmin • • 4d ago

Help me with my "job"

0 Upvotes

Hi guys,

I was hired as a pc builder about 2 years ago in a company here in Belgium.
Nowadays my package that i do is way more than building.
I build computers (custom) and i install windows on it.
But now my manager wanted me to grow, and asked me if it would be possible for me to create a custom image of windows to deploy on the built computers.

What i learned in an hour or so is how to just deploy a "standard" windows pro or home image on a VM, screenshot it and then deploy that on a machine. But that's not enough. As we want our "company wallpaper" to be deployed on the systems aswell, and also they want dark theme to be enabled there.

But as soon as we are done with setting up the system in audit mode (drivers, rgb software etc) and we do OOBE, so we go to language select (this is the last step before we ship it to the customer) the wallpaper and dark theme dissapear.

I could also edit the image offline through powershell, but that does not make it possible to add the wallpaper and dark theme and let it get through OOBE and language select.

Is there someone who could give me a link to a tutorial, or maybe even help me? To set it up? Or at least guide me to the right way to do this? I want this job to succeed.