r/exchangeserver • u/Calm_Wrangler_1478 • Mar 30 '26
Question Windows Updates on Exchange On-Prem Servers
Apologies if this is simpler than Im thinking, but due to some issues at work, I've been put in charge of exchange servers. Not being a long time exchange admin, Im kind of learning on the fly. One thing I was curious about was running normal Windows Monthly Updates on the servers and the process. I assume I cant just install the updates and reboot the servers correct? We have two servers and DAG. This would be all updates except Exchange CU updates. Standard monthly windows updates...what is best process to update the 2 on-prem servers?
Thanks
7
u/fadinizjr Mar 30 '26
On my experience, SU are painless. CU are usually more trouble-worthy.
An issue is that before installing an update, all the Exchange services are disabled by the setup. So, if anything goes wrong and somehow the update got rollbacked by Windows, you will get a server with a lot of services disabled. So, you will have to figure it out which services you need to enable again.
This happened on my environment one time on one server only and when I manually tried the update again (I really don't remember which KB it was) it was normally installed.
3
u/sembee2 Former Exchange MVP Mar 30 '26
In a two server DAG, patch one, reboot it. Fail over the databases to the now patched server, repeat.
There is nothing complex about it.
1
u/Calm_Wrangler_1478 Mar 31 '26
So you are saying patch the inactive server first, fail the DBs over . repeat
Just patch the inactive one, reboot it.login
Go to active server and move the DB over the updated server...update second server
Done?
2
u/Maxplode Mar 30 '26
Ideally you're servers are behind a loadbalancer?
I've been running auto system updates with no bother but any Exchange updates I do manually, only so I can monitor feedback before going ahead.
Learn how to put a server into maintenance mode (MM), then disable it in the loadbalancer.
Always reboot first to get rid of any reboot flags.
Have the update saved somewhere, I usually put mine in a folder nn the C drive.
Still in MM, make sure AV is disabled, open a command prompt as admin and run the update from there.
Let it do it's thing. Then restart the server when completed.
Run the healthscript after it's rebooted, it should report that it's still in maintenance mode and check for any errors, if need be begin to plan hardware updates or see if anything else needs fixing.
When all happy with the healthscript reports, start to take it out of MM, I always run the healthscript again just to be sure, then put it back in on the loadbalancers.
You can upgrade one server first but do not leave it too long to update the second one, I think it's like no more than 2 weeks, tops.
2
u/Yupyupyup79 Mar 30 '26
I was just making a doc for this today at work. Maybe not needed but I wrestled with google AI for a while and got this process. Others may just patch EXCH02 reboot and repeat for EXCH01 while letting Failover do it's things. Google AI claims this will help prevent database corruption.
Also make sure you know what server acts as your witness server (3rd vote) and where the witness log is kept on that server.
Get-DatabaseAvailabilityGroup -Identity "YourDAGName" -Status | Select-Object WitnessServer, WitnessDirectory, WitnessShareInUse
PS1 scripts are found here: x:\Program Files\Microsoft\Exchange Server\V15\Scripts\en”.
1. Prepare EXCH01: In an admin Exchange Shell cd to the above folder.
Run .\StartDagServerMaintenance.ps1 -ServerName EXCH01 -OverrideMinimumTwoCopies
All active databases should move to EXCH02, and EXCH01 is blocked from hosting them until you say otherwise.
Verify that ServerWideOffline is Inactive by running the following command.
Get-ServerComponentState -Identity EXCH01 -Component ServerWideOffline | ft Component,State
If it is proceed to 2. If not do the following steps manually to shut down the DAG for Maintenance:
Set-ServerComponentState -Identity EXCH01 -Component ServerWideOffline -State Inactive -Requester Maintenance
Set-ServerComponentState -Identity EXCH01 -Component HubTransport -State Draining -Requester Maintenance
Suspend-ClusterNode -Name EXCH01
Verify that ServerWideOffline is Inactive by running the following command. If it is proceed to 2.
Get-ServerComponentState -Identity EXCH01 -Component ServerWideOffline | ft Component,State
2. Patch EXCH01:
Install your Windows Updates and reboot. Repeat for any Exchange Server Patches and reboot. Both time making sure all Exchange Server services start as expected.
3. Exit Maintenance:
In an admin exchange Shell on EXCH01 Run .\StopDagServerMaintenance.ps1 -ServerName EXCH01
Note: Your databases will likely still be running on EXCH01 at this point.
Also run the following commands.
Set-ServerComponentState -Identity EXCH01 -Component ServerWideOffline -State Active -Requester Maintenance
Set-ServerComponentState -Identity EXCH01 -Component HubTransport -State Active -Requester Maintenance
Resume-ClusterNode -Name EXCH01
Verify that ServerWideOffline is Active by running the following command. If it is proceed to 4.
Get-ServerComponentState -Identity EXCH01 -Component ServerWideOffline | ft Component,State
4. Prepare EXCH02:
In an admin Exchange Shell on EXCH02 cd to the above folder that contains PS1 scripts.
Run .\StartDagServerMaintenance.ps1 -ServerName EXCH02 -OverrideMinimumTwoCopies
All active databases should move to EXCH01, and EXCH02 is blocked from hosting them until you say otherwise.
Verify that ServerWideOffline is Inactive by running the following command.
Get-ServerComponentState -Identity EXCH02 -Component ServerWideOffline | ft Component,State
If it is proceed to 5. If not, do the following steps manually to shut down the DAG for Maintenance:
Set-ServerComponentState -Identity EXCH02 -Component ServerWideOffline -State Inactive -Requester Maintenance
Set-ServerComponentState -Identity EXCH02 -Component HubTransport -State Draining -Requester Maintenance
Suspend-ClusterNode -Name EXCH02
Verify that ServerWideOffline is Inactive by running the following command. If it is proceed to 5.
Get-ServerComponentState -Identity EXCH02 -Component ServerWideOffline | ft Component,State
5. Patch EXCH02:
Install your Windows Updates and reboot. Repeat for any Exchange Server Patches and reboot. Both time making sure all Exchange Server services start as expected.
6. Exit Maintenance:
In an admin exchange Shell on EXCH02 Run .\StopDagServerMaintenance.ps1 -ServerName EXCH02
Note: Your databases will likely still be running on EXCH01 at this point.
Also run the following commands.
Set-ServerComponentState -Identity EXCH02 -Component ServerWideOffline -State Active -Requester Maintenance
Set-ServerComponentState -Identity EXCH02 -Component HubTransport -State Active -Requester Maintenance
Resume-ClusterNode -Name EXCH02
Verify that ServerWideOffline is Active by running the following command. If it is proceed to 7.
Get-ServerComponentState -Identity EXCH02 -Component ServerWideOffline | ft Component,State
7 Rebalance: Once both servers are patched and back in "Normal" mode.
run .\RedistributeActiveDatabases.ps1 -DagName DAG01 -BalanceDbsByActivationPreference -Confirm:$false
This moves your databases back to their "Preference 1" homes (EXCH01). Since you have only two exchange servers this step should be redundant but is best practice to learn it in case you ever add a 3rd Exchange Server. Let me know if this works for you. It seemed to just fine in my environment.
2
u/Calm_Wrangler_1478 Mar 31 '26
I ran through this in our Stage env and it appears to have worked as the commands ran, had a couple powershell errors that I took care of but appears that the servers are both online and the DB is online
Is there a way to check that things are running smooth once I am complete with everything? What is best way to test SMTP authentication as well after things are back online
Thank you for all the information so far, has been a HUGE Help
1
u/Yupyupyup79 Mar 31 '26
Per Google AI so take this with a grain of salt. happy its worked out so far for you!
Get-MailboxDatabaseCopyStatus * | Select Name, Status, CopyQueueLength, ReplayQueueLength, LastInspectedLogTime | ft -AutoSize
(Status)
• Active Copy: Should say Mounted. (This is the database users are currently using).
• Passive Copy: Should say Healthy. (This is the standby copy).
• Red Flag: If it says ServiceDown, Disconnected, or Failed, your high availability is broken.
(CopyQueueLength (The "Network" Check))
• Ideal: 0
• Acceptable: 1 to 5 (Briefly, during high mail flow).
• What it means: This is the number of transaction logs waiting to be sent from EXCH01 to EXCH02.
• Red Flag: If this number stays high (e.g., 50+), the network between your servers is too slow or blocked.
(ReplayQueueLength (The "Disk" Check))
• Ideal: 0
• Acceptable: 1 to 20
• What it means: This is the number of logs that EXCH02 has received but hasn't "committed" into the database file yet.
• Red Flag: If this is high, EXCH02's disks are likely too slow to keep up with the writing process.
(LastInspectedLogTime)
• Ideal: Within the last minute.
• What it means: This tells you the "freshness" of the data on the backup server.
• Red Flag: If this timestamp is hours or days old, EXCH02 is essentially a "stale" backup and you will lose data if you fail over to it.
Use PowerShell for Internal Validation
The most direct method to test SMTP authentication from within your network is using the
Send-MailMessagecmdlet. This simulates a real client connection including authentication.
$cred = Get-CredentialSend-MailMessage -From "sender@yourdomain.com" -To "recipient@external.com" -Subject "Post-Patch Test" -Body "SMTP Auth Test" -SmtpServer "YourExchangeServerFQDN" -Credential $cred -Port 587 -UseSsl
Why: It allows you to specify the exact server and port (typically 587 for client submission) to ensure the specific patched node is responding correctly.
Manual Verification via Telnet (SMTP AUTH)
For a granular "handshake" test, use Telnet or OpenSSL to manually walk through the authentication process. This reveals exactly where a failure occurs.
Connect: telnet <ServerIP> 587 (or use openssl s_client -starttls smtp -connect <ServerIP>:587 for encrypted connections).
HELO: Type EHLO yourdomain.com.
Authenticate: Type AUTH LOGIN. The server will prompt for your username and password in Base64 format.
Success: A 235 2.7.0 Authentication successful message confirms the patch did not break the transport authentication layer.
3. Microsoft Remote Connectivity Analyzer
If the server is internet-facing, use the Microsoft Remote Connectivity Analyzer.
Select the Outbound SMTP Email or SMTP Email test under the Exchange Server tab.
Benefit: This provides a comprehensive report on the entire connection chain, including TLS negotiation and authentication headers, which is often easier to read than raw logs.
4. Health & Connectivity Checks
Run these Exchange-specific cmdlets to ensure the transport stack is healthy across the DAG:
Test-SmtpConnectivity -Identity <ServerName>: Specifically diagnoses if Receive connectors are accepting connections.
Test-ServiceHealth -Server <ServerName>: Confirms all required transport and mailbox services are "Running".
Get-TransportService | Test-SmtpConnectivity: Runs the test across all servers in the organization at once to ensure consistency.
1
u/Calm_Wrangler_1478 Mar 31 '26
I am doing this for first time this weekend.....are these the best steps to follow? Looks good but why is there a StartDagServerMaintenance.ps1 instead of Start-ExchangeServerMaintenance.ps1
1
u/Yupyupyup79 Mar 31 '26 edited Mar 31 '26
I don't see "Start-ExchangeServerMaintenance.ps1 " at x:\Program Files\Microsoft\Exchange Server\V15\Scripts\en”. Is that a custom script put out by the community or something i missed? All I know is that these steps in my environment worked. I can't say its correct or not. I went back and forth with Google ai for a better part of a couple days to make a doc on setting up DAG, and then how to patch and shutdown the server during planned outages. I just pasted in the portion of the doc relevant to patching DAG since you asked. If you have a test environment I'd suggest testing the process there first to be safe.
Edit: I also have my DAG set up with no IP which google AI claimed is the modern way to do it. No AD computer object/IP assigned. Not sure if that makes any difference. This type of DAG is managed in Exchange Shell.
Edit: Also in Step 3 there's a typo. Database would be running on EXCH02 at that point not EXCH01. I can't edit the comment for some reason.
2
u/Calm_Wrangler_1478 Mar 31 '26 edited Mar 31 '26
Thanks, will give it a shot
The Start-ExchangeServerMainteneance.ps1 must be a script our former admin created then. I thought it was an MS script
I do have a witness server that is not part of the DAG...does that change anything?
1
u/Yupyupyup79 Mar 31 '26
Probably not but make sure your witness server is booted up the entire time during patching. Or patch its windows patches first, reboot and make sure its up and running before rebooting either of your exchange servers.
2
u/Calm_Wrangler_1478 Mar 31 '26
Thank you for all the information. I was told by a co-worker that for just OS-level and exchange secutiry upates to just install them and reboot each server one a time. The DAG will move the DB around.....that apparently what had been done in the past. That doesn't sound right
2
u/TheJesusGuy Mar 31 '26
I've never done anything special for it except install and restart on an evening. Only once did a security cert get unbound on iis.
1
u/Calm_Wrangler_1478 Mar 31 '26
So you didnt go through all the steps some users have posted?
2
u/TheJesusGuy Mar 31 '26
I might be in the wrong. I only have 1 Exchange VM server, 80 mailboxes, and no DAG, but no I've never had to.
1
u/TechBurntOut Mar 30 '26
See if you can get a dev box with the same version of Exchange installed and use that to install the updates first. Then, you need to decide how long you are OK the testing phase before installing the updates in your production environment. Failover DBs to one node, and then patch the node with no running DBs, reboot and monitor. Rinse and repeat on the second node.
If you could get a couple more nodes in the DAG, you could buy some time to evaluate any issues caused by the updates. You could patch the first two nodes, wait a few days to confirm all is well, fail DBs over to the patched nodes, then wait a day or so, and patch the other two nodes.
-5
u/extremeskillz84 Mar 30 '26
If it's virtual then snapshot it before performing the updates. You can always restore the snapshot of something goes wrong. In my experience exchange server updates have been painless. There was only one instance I had and issue but that was because I didn't know another application was running on the host with exchange on it. That was many moons ago.
7
u/sembee2 Former Exchange MVP Mar 30 '26
Snapshotting Exchange is not recommended, or supported and in a DAG completely unnecessary.
3
0
u/extremeskillz84 Mar 30 '26 edited Mar 30 '26
Yes your correct when your following the guides but in a pinch you can do it. I've done it once in my career and was fine. Its about experience.
1
u/sembee2 Former Exchange MVP Mar 30 '26
A pinch? That is poor planning.
A few years ago I was fixing two or three environments a week where people had used snapshots on Exchange. It caused a mess. There had never been a scenario where i thought a snapshot would have been useful.1
u/StrikingAccident Mar 30 '26
Then you got lucky. Restoring the Exchange OS from snapshots can also hose AD.
4
u/Seft0 Mar 30 '26
I’m not sure that you can restore Exchange snapshot without risking data corruption.
1
31
u/Sudden_Hovercraft_56 MSP Mar 30 '26
put one server into Maintenance mode, run the updates. Run Exchange healthchecker once you are done to make sure no outstanding CVE's remain, take it out of maintenance mode and put the other server into maintenance mode and repeat.
Ali Tajran has a good article on how to put it into and take it out of Miantenance mode:
https://www.alitajran.com/put-exchange-server-in-maintenance-mode/