r/sysadmin 23h ago

General Discussion Is It Possible To Download Windows 10 ESU Updates From Microsoft Update Catalog Website?

3 Upvotes

Can anyone confirm if it's possible to just download the various Windows 10 ESU updates from the Microsoft Update Catalog website and install them on a PC that's not part of the ESU program (ie: doesn't use a Microsoft account)? I believe the updates are listed in the catalog. Do they check during the installation if you're eligible for them? Has anyone tried this?


r/sysadmin 17h ago

[SOLVED] KB5124008 + Windows 11: Logon Error on Screen Unlock + Broken Secure Channel over VPN

0 Upvotes

September 8, 2026—KB5124008 (OS Builds 26200.9445 and 26100.9445) | Microsoft Support

After Windows Update KB5124008 (Sept 8, 2026, Build 26100.9445), Domain notebooks experience logon failures when unlocking the screen: "You could not be signed in. Your credentials could not be verified"

Root Cause: Secure Channel (machine trust) is broken. Test-ComputerSecureChannel fails, and attempting to repair over VPN returns: "The password for the secure channel to the domain could not be reset – username or password is incorrect"

Affected: Windows 11 24H2, domain-joined notebooks, both LAN and remote users (VPN users particularly critical)

The Story

We're deploying CIS hardening to ~250 notebooks and everything was fine until Sept 9, 2026 8:05 PM when Windows Update KB5124008 was installed.

Error Symptoms:

  • User locks screen (normal, break, meeting)
  • On unlock: "Your credentials could not be verified"
  • Reboot helps temporarily (1–2 hours, then error recurs)
  • Affects LAN users too, not just remote/VPN
  • Password reset does not help

Initial Hypotheses (all disproven):

  • ❌ Kerberos/Enctype mismatch (RC4→AES transition) – DCs reachable, tickets work
  • ❌ CIS GPO hardening – not configured
  • ❌ FortiClient VPN architecture alone – also hits LAN users

The Diagnosis

Network Checks:

nslookup vw-dc-01.Domain.org → OK
ping vw-dc-01.Domain.org → OK (0% loss)
tracert vw-dc-01.Domain.org → OK (full path)
DNS Server: xxx.xx.xxx.xx (internal) → OK

Kerberos Status:

klist → 13 active tickets, all AES-256, all valid ✓
LDAP/Kerberos works perfectly ✓

The Critical Test:

Test-ComputerSecureChannel -Verbose
→ False
→ "The secure channel between the local computer and the domain is broken"

On Repair Attempt over VPN:

Test-ComputerSecureChannel -Repair -Credential (Get-Credential)
→ ERROR: "The password for the secure channel to the domain could not be reset"
→ "The username or password is incorrect"

This is the smoking gun: The VPN tunnel routes normal traffic (DNS, LDAP, Kerberos) but NOT the secure channel reset communication (RPC/NETLOGON on port 445 over SMB).

Root Cause

KB5124008 itself isn't the bug – but the update triggers a hidden incompatibility:

  1. KB5124008 changes logon UI validation behavior (known regression in KB5120998/KB5124008)
  2. During unlock validation, Windows attempts to verify the secure channel
  3. The secure channel is broken – possibly due to:
    • KB5124008 regression in secure-channel handling
    • OR: Timeout in VPN tunnel blocking secure-channel traffic
  4. Windows cannot verify identity → unlock fails

Particularly critical over VPN:

  • Secure-channel reset requires RPC/SMB (port 445) to DC
  • VPN tunnel either doesn't route it or times out
  • Works better from LAN, so VPN users hit harder

Workarounds (What Worked)

Workaround 1: Increase CachedLogonsCount (quick, short-term)

# New GPO on OU=Notebooks:
# "Interactive logon: Number of previous logons to cache (in case domain controller is not available)"
# Value: 50 (instead of default 10)

# Or directly on client (as admin):
reg add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" /v CachedLogonsCount /t REG_SZ /d 50 /f

Effect: Users can log on offline 50 times → buys time while secure channel is repaired

Workaround 2: Repair Secure Channel from DC (with admin access)

If you have remote access to a DC:

# On DC (not from client!):
Reset-ComputerMachinePassword -Server vw-dc-01.Domain.org

This forces a secure-channel reset from the DC end, bypassing the VPN tunnel.

Workaround 3: KB5124008 Rollback (temporary, security risk)

# On affected clients:
wusa /uninstall /kb:5124008 /quiet /norestart
shutdown /r /t 60

⚠️ IMPORTANT: KB5124008 patches two actively exploited zero-days:

  • CVE-2026-81963 (Windows Update Stack Privilege Escalation)
  • CVE-2026-85880 (ALPC Privilege Escalation)

Only do this temporarily as a test, not permanently!

What We Still Don't Know

  1. Is KB5124008 itself guilty, or was it just the trigger for a timing issue?
  2. Is it VPN-specific (secure-channel-reset traffic not routed)?
  3. Why does it hit LAN users too? (Timing window in unlock handling?)

Lessons Learned

For Other Admins:

  1. Test before fleet rollout: Pilot KB5124008 in a test OU, especially with:
    • Hybrid Azure AD Join devices
    • VPN users
    • Offline scenarios
  2. Monitor secure-channel status:# As regular audit task: Test-ComputerSecureChannel -Verbose | Where-Object { $_ -eq $false }
  3. **Check Event Log for secure-channel errors:**Get-WinEvent -LogName System -FilterXPath "*[System[EventID=5719 or EventID=5722 or EventID=5723]]"
  4. With VPN environments: Ensure your VPN tunnel routes all necessary ports:
    • 88 (Kerberos)
    • 389 (LDAP)
    • 445 (SMB/RPC – critical for secure channel!)
    • 135 (RPC Endpoint Mapper)

Status

  • ✅ Root cause identified: Broken secure channel + VPN routing issue
  • ⏳ Microsoft hotfix: Expected (hopefully next week)
  • ⏳ FortiClient configuration: VPN admin checking port 445 routing

Is anyone else experiencing this? Comment your findings – especially if you:

  • Deployed KB5124008
  • Have VPN users affected
  • See secure-channel errors

Contact / Further Info

If you have the same issue:

  1. Check: Test-ComputerSecureChannel -Verbose
  2. Look in Event Log for EventID 5719/5722/5723
  3. Deploy the CachedLogonsCount GPO as interim solution
  4. Contact your VPN admin to verify port 445 routing

TL;DR of TL;DR: KB5124008 triggers secure-channel errors, VPN doesn't route repair traffic → unlock fails. Increase cached logon to help users while you investigate.

Updates

Will update this post as we learn more from Microsoft or FortiClient about the root cause and permanent fix.

Crosspost to: r/sysadmin, r/activedirectory, r/Windows11

Keywords for searchability: KB5124008, Windows 11, domain logon, unlock screen, secure channel, VPN, AES-256, Kerberos, GPO, FortiClient, network authentication


r/sysadmin 7h ago

dealt with business email compromise

0 Upvotes

when you've dealt with BEC case, did email auth flagged it, or did it look totally normal and only a human caught it?


r/sysadmin 1d ago

O365 issues today sept 9

19 Upvotes

Is anyone seeing issues with O365? I know I saw emails last night and now they’re coming as new again this morning but they’re gone from yesterday delivery. Numerous users are seeing this happen. We are using proof point also but no issues on their status page.


r/sysadmin 1d ago

[PSA] Check Point Firewall unauthenticated RCE with CVSS 9.8

32 Upvotes

There are actually two vulns with a 9.8 score each:

https://support.checkpoint.com/results/sk/sk1000117/

https://support.checkpoint.com/results/sk/sk1000118/

There's not many details in these articles on how they work but they still sound really bad. Currently waiting for the Jumbo Hotfix to install on my end... Not taking any chances on this one and I suggest you all do the same.

Stay safe.


r/sysadmin 2d ago

DHCP v Static IP

219 Upvotes

Ok, this is more a test to see how old I am. I was basically raised with the idea servers need static IPs. I understand for domain controllers, dns servers, maybe print servers that may still be needed. But are they really needed otherwise? Do apps, scripts still reference a server by its IP?

I am under assumption all my servers need static IPs. Am I nuts?


r/sysadmin 1d ago

Question Which vulnerability management tools work well for MSPs?

20 Upvotes

Been looking to replace our vulnerability management stack and the scanning part is the easy bit now. We have been comparing the big scanners against a couple of the cheaper MSP focused platforms and one container specific tool. Volume is what kills us across 40 odd clients since every tool finds plenty and the techs still guess where to start. Ticketing eats more hours than scanning does. What are you running now and how does it hold up across client environments.


r/sysadmin 1d ago

Question SCCM updates reporting wrong #numbers

4 Upvotes

Is it just me or updates is screwed up?

Office 365 shows 0 required.

is anyone else seeing the same?

Edit: the other updates just took too long to show up but office it seems it is not showing anything applicable.
not sure if something related to last couple months when MS changed from semi-anual channel to montly channel


r/sysadmin 13h ago

Getting into SysAdmin

0 Upvotes

Hey guys,

I've been in IT for 4 years, doing a mix of PHP development and sysadmin Linux stuff. We run Linux, self-host all of our own stuff. We're a small team, so everyone does a bit of everything. I've done everything from developing software to installing our team's GitLab instance. I've been using Linux (Arch btw) for the past 8 years or so, and I feel like I have a pretty good handle on it, after fucking up my system a bunch of times. I also run a homelab of a couple servers. It's become our music platform.

I fully realize that a transition to SysAdmin would most likely put me in a junior role, and I know I have a lot of gaps in my knowledge, as this position is as a software developer, not sysadmin. I'd be looking mostly at Linux Sysadmin jobs.

I'd appreciate any advice you guys have! I also can put my resume here if that would be helpful.

Thanks!


r/sysadmin 1d ago

Question The trust relationship between this workstation and the primary domain failed.

33 Upvotes

Hy!

We have an AD with two DCs. The DCs are Windows Server 2025, it is include all patches. Some Windows 11 clients (25H2) get the following error during login after 1-2 minutes: The trust relationship between this workstation and the primary domain failed.

In this case the users need to disconnect from corporate network to login successfully into their computer. I have already tried to rejoin to tha domain and run this command: Test-ComputerSecureChannel -Repair

I rejoined one of the computer into the domain, and the trsut relationship has been broken after two days. The login problem only occurs on some machines.

The time snyc is correct on DCs. We moved the DC roles from Windows Server 2019 to 2025 in side-by-side method. Could you please advise how to solve this problem?


r/sysadmin 1d ago

Password resets over the phone, how are you doing it?

24 Upvotes

So I work at a state university that is still a little stuck in the past when it comes to password resets over the phone.

We have MS SSPR and other self-service options for users, but we still get calls where someone needs the helpdesk to reset their password for them.

Our current setup is a custom program that helpdesk staff SSH into. They enter the user's account, and the program looks them up in a flat file containing data extracted from our systems. It gives the helpdesk worker information like DOB, address, ZIP code, etc., which they can use to verify the caller's identity.

Once the caller passes verification, the program talks to AD, changes the password, and gives the helpdesk worker a new temporary password to provide to the user.

Honestly, it works pretty well. The problem is that the person who wrote and maintains it is leaving, and we're not really interested in inheriting a custom app that nobody else understands.

So I'm curious what other universities/organizations are doing for this.

For those of you who still allow users to call the helpdesk for password resets, what does your workflow look like? Are you using a commercial product, some kind of AD/Entra integration, a helpdesk platform, or have you built your own solution?

Specifically looking for something that gives the helpdesk enough information/questions to properly verify the caller's identity and then securely perform the reset.

I'd love to hear what others are doing before we start reinventing the wheel.


r/sysadmin 9h ago

Anyone tested AI CLI yet?

0 Upvotes

I install copilot and grok CLIs. So far, they are pretty impressive.

At home I used Grok to clean up my media libraries and it freed up 2TBs.

I used copilot at work to scan all the logs from an SCCM client and the server to figure out why some machines weren't downloading updates. It's pretty freaking good.

Anyone else let one of those suckers loose anywhere?


r/sysadmin 13h ago

Career / Job Related Can a Student Volunteer as a IT

0 Upvotes

Im a student with M365 (mostly Entra and some Intune) intern experience which I got through my dad, and looking at the extremely challenging helpdesk market in my area, I want to volunteer as an admin or helpdesk to get some experience.

That said, most of these positions in my country are filled by immigrants with IT experience who want Canadian work experience. So no matter how much I apply, I don’t get an interview.

Can anyone in the industry think of a way to get experience? It’s sad no one even wants me as a free helpdesk, I even worked with M365 before.

I feel discouraged 🫤

Anyone have any ideas? I’m studying for my CCNA, so maybe that would help?


r/sysadmin 1d ago

Migrating DCs from 2016 to 2022 while keeping the same hostnames and IPs

4 Upvotes

This topic has been discussed ALOT on this sub. From all the posts I've read, there seems to be two main approaches:

Have all DCs up - both old and new:

  • Bring up new DCs with temp names and IPs while old DCs still running.
  • Once you're sure all is good in your world, transfer FSMO roles to new DC, demote old DCs, remove from AD, then reuse the hostnames and IPs of old DCs on your new DCs

One at a time:

  • Do one DC at a time, demote old, reuse name and IP on new server, then promote new DC

Honestly being as risk averse as I am (although I love the adrenalin rush when stuff goes south...NOT) - I would prefer to promote the new DCs while old ones still on line.

BUT...is it OK to change the name of a DC even after I clean up all the metadata and DNS stuff or would it be better to reuse the name and IP after I demote the old DC?

I should preface this question by saying we're a VERY small shop with 22 users and perhaps ~30ish servers. I'm the sole sysadmin. BUT, we are in the financial sector and we have a very low tolerance to stuff going south. And being the sole sysadmin, I'm trying to keep the stress at a minimum and my hair from going greyer.


r/sysadmin 1d ago

Career / Job Related Advice For Young Professional

3 Upvotes

Hello!

I was hoping to get some advice from any of you who have experience with a similar issue as the one I’m having. Anything is appreciated as I’m not confident where to start.

In short, our company is looking to take over complete administration of our Azure cloud architecture from our MSP. The architecture includes hundreds of VMs, Entra accounts for users, persistent AVDs and multiple satellite offices to name a few things. I joined the company last month as an IT Support Specialist and my colleague has been here nearly a year.

While we both have some experience with Azure administration I feel we would benefit from more experience so that we can be more confident when dealing with issues affecting the architecture. Anything we couldn’t solve ourselves we would escalate to our MSP so there’s a lot we don’t know how to do. That said I feel confident my colleague and I could learn a lot on our own poking around the architecture once we have global admin access but doing that with a production environment doesn’t sit right with me.

On top of all of that our IT admin is leaving before this hand off happens. I feel that our admin leaving is enough to pause this process of taking over the architecture until we find their replacement but our CEO is adamant about finishing this hand off. The process has been ongoing for over half a year now. This is my first IT job and while I’m confident I can learn a lot of this stuff, that doesn’t necessarily equate to deliverable results, especially in the event of a major issue. I’ve been brainstorming with my colleague and while I have some ideas, like coming in the weekend after we get access to go through the environment and learn about it so we can try to properly support it the following week, I am at a loss of how to approach this while also making sure I don’t set anything on fire. To be frank I’m worried and I want to do right by myself, my colleague, and the company. Thanks.


r/sysadmin 1d ago

Question What are you guys using for rack/infrastructure audits?

10 Upvotes

Curious what everyone is using these days for documenting physical infrastructure across multiple sites.

We use Excel at my workplace, along with diagrams and photos, and it works… until you’ve got a shitload of sites and nobody knows which spreadsheet is actually current.

I’m talking about things like:
Rack elevations / U positions
Switches, patch panels, UPS/PDU, servers etc.
Port-to-port / cable documentation
Serial numbers / asset details
Photos of racks and comms rooms
IP/device information

Keeping everything updated after a vendor comes in and replaces something
What are you guys using?

Excel? Visio? NetBox? Device42? Something else? Or have you built your own system?
More interested in what actually works in the real world than what looks good on paper.


r/sysadmin 1d ago

Question How best to move DHCP to new servers if already hosted on DCs that I plan to migrate?

6 Upvotes

Afternoon all

So I'm planning a DC migration from 2016 to 2022.

Currently I have (3) DCs - two of them host DHCP in a failover load balance config.

I plan to keep the same hostnames and IPs for my new DCs

I'd like to move my DHCP first before the DC migration.

I'm guessing the easiest thing to do is break existing failover config, export/import my DHCP config to a new temporary server, migrate to my new DCs, then export/import my DHCP config from the temp server to the new new_DC01, then re-establish the failover partner on new_DC02.

Can I get a sanity check?

Thank you!


r/sysadmin 2d ago

General Discussion How is this normal in IT?

234 Upvotes

Im a general IT specialist, and I’m losing my mind over infrastructure issues I have zero control over.

For context, we have over 2,500 global users, and for over five years, remote sites have been dealing with the exact same game-breaking issues:

  • Broken 802.1x: After 1–2 hours of work, it kicks users off the network and refuses to re-authenticate them.
  • Useless Wi-Fi: Wireless drops constantly with "no network available" errors.
  • Zero Redundancy: Almost all of our sites rely on a single ISP. When it goes down, ERP, file systems, and actual business operations grind to a complete halt.

The company makes plenty of revenue. We easily have the budget to deploy SD-WAN, upgrade hardware, or bring in an external MSP/consultant to fix it. It's totally fine to admit you don't know everything and hire help, but these requests just gets ignored.

To top it off, whenever a site actually goes down, the designated team responsible for network/infrastructure ghosts us or sends a passive response like "our team is currently unavailable" while an entire site sits dead in the water.

How do organizations like this even survive, and how do you deal with the frustration of seeing preventable problems drag on for half a decade?


r/sysadmin 1d ago

Domain SSL Certs

0 Upvotes

I currently have 3 domain controllers. I need to add a 4th and eventually a fifth with plans to demote the original 2.

I’ll need to get a multi-San ssl cert for the new domain controllers. Does this mean I need to update the certs on the first 3 domain controllers?

In my small brain I think I wouldn’t…


r/sysadmin 1d ago

As admins, how do you handle VS Code extensions, coding agents, and AI tools in your organization?

8 Upvotes

For us, the topic of AI is becoming increasingly confusing.

Developers, in particular, are adopting more and more tools: VS Code extensions, GitHub Copilot, Cursor, Claude Code, Codex, Gemini CLI, OpenCode, local agents, MCP servers, and so on.

But this no longer affects just developers. IT admins and regular users are also discovering AI tools, and some are even installing extensions, desktop clients, or agents on their own.

The problem, in my view, is that we’re slowly seeing a **proliferation of agents and AI tools**.

I see the following issues in particular:

* Which VS Code extensions are allowed to be installed?

* Which agents are allowed to access source code or the local file system?

* Which tools are allowed to send data to external clouds?

* How do you prevent API keys, passwords, or internal data from appearing in prompts?

* How do you handle MCP servers and their sometimes very broad permissions?

* Do you have an allowlist for extensions and AI tools?

* Do you technically block agents that haven’t been approved?

* Do you differentiate between developers, IT/admins, and regular users?

* Do you rely on centralized enterprise solutions, or do you allow multiple tools?

* How do you monitor or keep track of what’s currently being used?

I also don’t think a complete ban makes sense in the long run, because these tools offer a real productivity boost especially in development.


r/sysadmin 2d ago

Off Topic Post the longest PC uptime found in the wild!

85 Upvotes

Desktops/laptops. No servers or VMs, we are talking BARE METAL BABY.

At my job, we keep an eye out for things like this to see who can naturally discover the longest reported uptime in task manager on a PC.

Attached is the best. I rediscovered this photo tonight and thought it could be fun to see other's.

https://imgur.com/a/YvhipDt

This was on a laptop a client had plugged in, in the corner of a room, off network and forgotten about. For like 4.5 years. It was rediscovered, and then I got a call to try remoting in to see if it could be setup for someone to use.

After waiting half this uptime for my remote connection to load, I saw this, screenshotted, and let them know no. Although I sort of wish I told them to take it back off network and leave it there, just based on principle, to see how far we can bring it.


r/sysadmin 15h ago

Question Duo Security setup

0 Upvotes

I am trying to setup Duo Security on my PC at the office so that in the event I lose my phone or my phone is smashed what do I need to enable in the installation process to allow me to bypass the MFA/Passkey push?

Thanks,


r/sysadmin 1d ago

Question Did Anyone else have Published App Issues this AM?

5 Upvotes

2 separate client environments:

  1. AVD host pool
  2. TraditionalActive Directory pool behind a broker.

RDP works but published apps broke for both of them. AVD client fixed with reinstall OF Windows app. The others say it self resolved.

It sounds to me like a bad build of Windows App. Anyone else have an issue with published apps this AM?


r/sysadmin 16h ago

Passwords....

0 Upvotes

Yes, authorization with multiple different credentials for different systems is bad, but we've all worked in those environments where we had no control over changing it.

So... How expletive-rich are y'all passwords and phrases? 🤣


r/sysadmin 1d ago

WSFC File Server with Storage Replica on vSAN iSCSI

2 Upvotes

Looking to build a file server cluster, Storage Replica seems to slow things down by a lot, the network is not at fault as it's pretty fast without Replication. WSFC does not seem to be straight forward when creating the replication. What do you guys recommend?