Gracefully Patching Exchange 2016

Though most businesses these days are moving to cloud providers for their email services, there are still quite a lot of on premise installations of Exchange 2016 out there. When it comes time to apply Windows updates, upgrade your Exchange servers to the latest CU or just perform a reboot, there is a procedure that you should follow to avoid disrupting client access.

For the purposes of this post, we will be upgrading 2 Exchange 2016 servers (EXCH1 and EXCH2) in a DAG that is fronted by a load balancer.

We will start with EXCH1. Log into your load balancer and set any virtual services you have for the DAG to drainstop and disable any connections to EXCH1 (Typically there would be SMTP and HTTPS virtual services). This will force any subsequent client connections to EXCH2.

Log into EXCH1 and open the Exchange Management Shell as Administrator.

We will start by setting the Hub Transport Service to Draining to stop it accepting any more messages.

Set-ServerComponentState EXCH1 –Component HubTransport –State Draining –Requester Maintenance

Now we will redirect any queued messages to EXCH2.

Redirect-Message -Server EXCH1 -Target EXCH2

And suspend EXCH1 from the DAG.

Suspend-ClusterNode –Name EXCH1

Next step is to disable Database Copy Auto Activation and initiate the move of any active databases on EXCH1 over to EXCH2.

Set-MailboxServer EXCH1 –DatabaseCopyActivationDisabledAndMoveNow $true

We want to confirm that the DatabaseCopyAutoActivationPolicy setting is set to Unrestricted.

Get-MailboxServer EXCH1 | Select DatabaseCopyAutoActivationPolicy

And now set it to Blocked to prevent any of the Databases from becoming Active.

Set-MailboxServer EXCH1 –DatabaseCopyAutoActivationPolicy Blocked

We will now check that all of the Active Databases on EXCH1 have moved to EXCH2.

Get-MailboxDatabaseCopyStatus -Server EXCH1 | Where {$_.Status -eq "Mounted"}

It may take a while for the Active databases to move, you can always log into the Exchange Admin Centre and manually move them if you are in a hurry.

Once the Active databases have all been moved we will put EXCH1 into maintenance mode.

Set-ServerComponentState EXCH1 –Component ServerWideOffline –State InActive –Requester Maintenance

At this point I usually like to reboot the server to free up resources and clear any pending reboot statuses.

After the reboot, log back in and reopen the Exchange Management Shell as Administrator.

Check the status of the PowerShell Execution Policy by running the following:

get-executionpolicy –list

Ensure that the MachinePolicy and UserPolicy are set to Undefined (Refer to KB981474 to fix if they aren’t).

You are now free to install any required Windows Updates or apply the latest Exchange CU. Always review the accompanying ReadMe notes to ensure there will be no issues in your environment.

After you have finished the updates and performed a reboot it is time to get things up and running again.

Log back in and reopen the Exchange Management Shell as Administrator.

Remove the server from Maintenance mode.

Set-ServerComponentState EXCH1 –Component ServerWideOffline –State Active –Requester Maintenance

Resume the server in the DAG.

Resume-ClusterNode –Name EXCH1

Reset the DatabaseCopyAutoActivationPolicy setting to Unrestricted.

Set-MailboxServer EXCH1 –DatabaseCopyAutoActivationPolicy Unrestricted

Reset the DatabaseCopyActivationDisabledAndMoveNow setting.

Set-MailboxServer EXCH1 –DatabaseCopyActivationDisabledAndMoveNow $false

Set the Hub Transport service to accept email messages.

Set-ServerComponentState EXCH1 –Component HubTransport –State Active –Requester Maintenance

Log back into your Load Balancer and re-enable the virtual services for EXCH1 that you disabled previously.

Now we will repeat the process for EXCH2.

Log into your load balancer and set any virtual services you have for the DAG to drainstop and disable any connections to EXCH2 (Typically there would be SMTP and HTTPS virtual services). This will force any future connections to EXCH1.

Log into EXCH2 and open the Exchange Management Shell as Administrator.

We will start by setting the Hub Transport Service to Draining to stop it accepting any more messages.

Set-ServerComponentState EXCH2 –Component HubTransport –State Draining –Requester Maintenance

Now we will redirect any queued messages to EXCH1.

Redirect-Message -Server EXCH2 -Target EXCH1

And suspend EXCH1 from the DAG.

Suspend-ClusterNode –Name EXCH2

Next step is to disable Database Copy Auto Activation and initiate the move of any active databases on EXCH1 over to EXCH2.

Set-MailboxServer EXCH2 –DatabaseCopyActivationDisabledAndMoveNow $true

We want to confirm that the DatabaseCopyAutoActivationPolicy setting is set to Unrestricted.

Get-MailboxServer EXCH2 | Select DatabaseCopyAutoActivationPolicy

And now set it to Blocked to prevent any of the Databases from becoming Active.

Set-MailboxServer EXCH2 –DatabaseCopyAutoActivationPolicy Blocked

We will now check that all of the Active Databases on EXCH1 have moved to EXCH2.

Get-MailboxDatabaseCopyStatus -Server EXCH2 | Where {$_.Status -eq "Mounted"}

It may take a while for the Active databases to move, you can always log into the Exchange Admin Centre and manually move them if you are in a hurry.

Once the Active databases have all been moved we will put EXCH2 into maintenance mode.

Set-ServerComponentState EXCH2 –Component ServerWideOffline –State InActive –Requester Maintenance

At this point I usually like to reboot the server to free up resources and clear any pending reboot statuses.

After the reboot, log back in and reopen the Exchange Management Shell as Administrator.

Check the status of the PowerShell Execution Policy by running the following:

get-executionpolicy –list

Ensure that the MachinePolicy and UserPolicy are set to Undefined (Refer to KB981474 to fix if they aren’t).

You are now free to install any required Windows Updates or apply the latest Exchange CU. Always review the accompanying ReadMe notes to ensure there will be no issues in your environment.

After you have finished the updates and performed a reboot it is time to get things up and running again.

Log back in and reopen the Exchange Management Shell as Administrator.

Remove the server from Maintenance mode.

Set-ServerComponentState EXCH2 –Component ServerWideOffline –State Active –Requester Maintenance

Resume the server in the DAG.

Resume-ClusterNode –Name EXCH2

Reset the DatabaseCopyAutoActivationPolicy setting to Unrestricted.

Set-MailboxServer EXCH2 –DatabaseCopyAutoActivationPolicy Unrestricted

Reset the DatabaseCopyActivationDisabledAndMoveNow setting.

Set-MailboxServer EXCH2 –DatabaseCopyActivationDisabledAndMoveNow $false

Set the Hub Transport service to accept email messages.

Set-ServerComponentState EXCH2 –Component HubTransport –State Active –Requester Maintenance

You can run some tests to ensure that everything has started up OK.

Check that the cluster nodes have all come up OK.

Get-ClusterNode

Check that all of the required services are running OK.

Test-ServiceHealth

Test MAPI connectivity to each server that is hosting an active database.

Test-MAPIConnectivity –Server exch1

Test-MAPIConnectivity –Server exch2

Check the DAG Copy Status Health.

Get-MailboxDatabaseCopyStatus * | sort name | Select name,status,contentindexstate

Check Replication Health.

Get-DatabaseAvailabilityGroup | select -ExpandProperty:Servers | Test-ReplicationHealth

Check the Database Activation Policy is set to Unrestricted.

Get-MailboxServer EXCH1 | Select DatabaseCopyAutoActivationPolicy

Get-MailboxServer EXCH2 | Select DatabaseCopyAutoActivationPolicy

Check that the Server Component states are set to Active.

(Get-ServerComponentState -Identity EXCH1 -Component ServerWideOffline).LocalStates

(Get-ServerComponentState -Identity EXCH2 -Component ServerWideOffline).LocalStates

Job done!

The beauty of this method is that you should be able to perform upgrades and updates during business hours with out disrupting any clients.

9 Comments on "Gracefully Patching Exchange 2016"


  1. How is the patching procedere for a single Exchange 2016 on Windows Server 2016 without DAG? Do I need Maintenance Mode here too or can I just patch the system with MS Updates or via WSUS in the night and reboot as we did with the old Exchange 2010 single Server?

    Reply

    1. Hey Axel

      With a single server you can just apply the various OS and Exchange patches without following this procedure however it probably wouldn’t hurt to set the HubTransport component into the Draining state and then putting it into Maintenance mode so that you are controlling the exact moment that your Exchange server stops accepting messages and client requests (and when it begins accepting them when you have finished the patching).

      Regards

      Kirin

      Reply

  2. I got an IP-less DAG with two servers that I am looking after now, would these script work as there is no AAP?

    Reply

    1. Hi John

      I honestly don’t know, I cant see why not, would be curious to see how it goes in your environment.

      Regards

      Kirin

      Reply

  3. Under the installation directory something like “C:\Program Files\Microsoft\Exchange Server\V15\Scripts” you will find a usefull script named StartDagServerMaintenance.ps1 it will all that and more Patch or upgrade the node once you are done just run a second script located at the same location StopDagServerMaintenance.ps1 to bring back the node out of maintenance the same procedure to upgrade the remaining nodes ( Dag members).

    Reply

  4. What about when you have 2 DAGs in the environment? How can you run this for multiple servers across different DAGs executing the script from one server?

    Reply

  5. After running (Get-ServerComponentState -Identity EXCH1 -Component ServerWideOffline).LocalStates I found that all my components were set by a different requester. The name was “Functional” So I had to change the line to activated it to read

    Set-ServerComponentState TN-JHB-EMS01 -Component ServerWideOffline -State Active -Requester Functional

    That was also done for the hub transport service.

    After that all the services came up.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *