Veeam Backup & Replication – Exchange 2010 DAG Issue

I recently experienced an issue with a Microsoft Exchange 2010 Database Availability Group (DAG) failing over during a Veeam Backup & Replication job. The issue was occurring due to the snapshot committal process in VMware, which causes a brief pause in virtual machine I/O. This pause was causing the DAG member to lose sight of the file share witness, which in this case was housed on the customer CAS server, and subsequently fail over.

The resolution to this issue was to increase the CrossSubnetThreshold and CrossSubnetDelay of the cluster. The CrossSubnetThreshold specifies how many heartbeats can be skipped before the cluster fails over and the CrossSubnetDelay specifies the heartbeat interval. The threshold you set for both of these properties can depend on many factors, for example the speed of your underlying storage array or the size of the virtual machine that be being snapshot. In my case I needed to set both values to their maximum. This can be performed by carrying out the following:

1. Navigate to Start -> Administrative Tools and launch Windows PowerShell Modules

2. When the Powershell Window opens please enter the following command:
 
$cluster = Get-Cluster; $cluster.CrossSubnetThreshold = 10; $cluster.CrossSubnetDelay = 4000
 
3. Once the command has completed please run the following and ensure that the CrossSubnetDelay and CrossSubnetThreshold are set to 4000 and 10.
 
Get-Cluster | fl *

4. Re-run your Veeam backup job and see if the cluster fails over. If the backup completes correctly you can they reduce the CrossSubnetDelay and CrossSubnetThreshold to find the optimum values.

That’s it, your done.

Microsoft Exchange 2010 – Migration Mail Flow Issue

I recently experienced an issue when sending e-mails from an Exchange 2003 mailbox to an Exchange 2010 mailbox during a 2003 to 2010 migration. Messages could be successfully sent from Exchange 2010 mailboxes to Exchange 2003 mailboxes but not the other way around, the messages would simply queue on the Exchange 2003 server. After a period of investigation it appeared this issue was occuring due to a smart host not being set against the SMTP connector in the Exchange System Manager on the Exchange 2003 server. To resolve the issue a smart host was configured on the SMTP connector via ESM to be the customers internet service providers upstream mail relay. The steps taken to resolve this issue are detailed below:

1. Connect to your Microsoft Exchange 2003 server and open the Exchange System Manager.

2. In the Exchange System Manager expand Servers -> Server Name -> Connectors.

3. Right click your SMTP connector and select properties.

4. On the general tab check the “Forward all mail through this connector to the following smart hosts” radio button and enter your internet service providers upstream mail relay, for example smtp.myisp.co.uk.

5. Test mail flow between the Exchange 2003 and Exchange 2010 environments.

That’s it, hopefully your migration mail flow issues will now be resolved.

Microsoft Exchange 2010 – Cannot Send E-Mail To A Mail Enabled Public Folder

I was recently assisting a colleague with an issue he had experienced with a Microsoft Exchange 2003 to Microsoft Exchange 2010 migration. The migration had completed correctly, however for an unknown reason you could not send e-mail to any mail-enabled public folders, despite whether these were created newley from within the Exchange Management Console or were replicated as a part of the migration. All of the mail-enabled public folder properties were correct, including permissions, and Exchange 2010 Service Pack 1 had also been applied. When e-mailing a mail-enabled public folder from an external network no NDR was produced, however when e-mailing from the internal network the following NDR was returned:

#554 5.2.0 STOREDRV.Deliver.Exception:ObjectNotFoundException; Failed to process message due to a permanent exception with message The Active Directory user wasn’t found. ObjectNotFoundException: The Active Directory user wasn’t found. ##

After performing some research it turned out this is a known problem with Exchange 2010 migrations as detailed here. The issue occurs due to the legacy administrative group (First Administrative Group) being empty following the migration. To resolve this issue perform the following actions:

1. Open ADSI Edit on either a domain controller or your Microsoft Exchange 2010 server.

2. Navigate to the “CN=Servers” ADSI attribute by locating the below path, I have also included a screenshot of the location to help identify the attribute:

CN=Servers,CN=First Administrative Group,CN=Administrative Groups,CN=First Organization,CN=Microsoft Exchange,CN=Services,CN=Configuration,
DC=domain,DC=local

ADSI Exchange Severs 300x279 Microsoft Exchange 2010   Cannot Send E Mail To A Mail Enabled Public Folder

3. Right click the “CN=Servers” container and select delete. Click OK when prompted in order to confirm the action. Note: Do not delete the top level container “First Administrative Group”, this is against Microsoft best practices and may have a negative affect on your Exchange organisation.

4. Ensure replication of your Active Directory database has occurred to all domain controllers and then attempt sending an e-mail to one of your mail-enabled public folders. You should now find e-mail makes it way to these folders as expected.

For more information on this issue, please see the following URL: http://msexchangeteam.com/archive/2010/05/05/454821.aspx

Microsoft Exchange 2010 – SSL Certificates

I recently installed my first third party certificate into an Microsoft Exchange 2010 environment. This process is almost identical to how it is performed in Exchange 2007, however the PowerShell commands differ very slightly. Microsoft Exchange 2010 does actually support certificate requests and installations directly through the Exchange Management Console, however I’m not much of a fan of this, it has a slightly clunky feeling to it much like the Small Business Server 2008 implementation of wizard driven certifcate requests and installations.

For this certificate installation I was using a Go Daddy UCC 5 Domain certificate, which is more than adequate for Exchange 2010. You may see Exchange certificates branded as UCC (Unified Communications) or SAN (Subject Alternative Name), these are essentially the same, just different vendors choose to brand them differently. As a rule of thumb, always go for at least a five domain Unified Communications certificate otherwise you will find yourself in a pickle when it comes to adding subject alternative names. The following section of this post details the steps required to generate your CSR and install your certificate into your Exchange 2010 environment.

1. The first step of this process is to generate the CSR that will be used to tell your SSL vendor all about your environment. To make life a little easier, DigiCert have created a web based tool that will compile the required CSR PowerShell command for you, which you can find here

Hopefully once you have filled out the required fields in the DigiCert tool, you should have something very similar to the below screenshot. Please pay attention to the Subject Alternative Names used and the order that they have been entered in.

DigiCert CSR 300x159 Microsoft Exchange 2010   SSL Certificates

Please note that where contoso.com is your public domain name, and where contoso.local is your local domain name. Mail.contoso.com would the public DNS A record that is pointed to your internet endpoint, for example the global IP address of your router or firewall.

2. Once you have filled out of the required fields, click the generate button and then copy the generated PowerShell command to your clipboard. Open the Exchange Management Shell (EMS) on your Exchange 2010 server and paste the copied PowerShell command into the EMS and press return. Once this has completed, navigate to the C:\ drive on your server and open the generated .csr file in notepad. The content of this notepad file is the information you need to submit to your SSL vendor when they request CSR information.

3. When your certificate has been generated, download it from your SSL vendors website onto the root of the Exchange 2010 servers C:\drive. Once this has been performed, we need to action a PowerShell command to both import the new certificate and append this certificate to all Exchange 2010 services.  The PowerShell command you need to run is the following:

Import-ExchangeCertificate -FileData ([Byte[]]$(Get-Content -Path c:\your_certificate.cer -Encoding byte -ReadCount 0)) | Enable-ExchangeCertificate -Services IIS,POP,IMAP,SMTP

In the above command, please edit the -Path attribute to read the file location of where you have stored your downloaded certificate.

4. To verify the certificate installation, browse to Outlook Web Access from an external source, so for example to https://mail.mydomain.com/owa and ensure that the correct certificate is being utilised by clicking the padlock icon in Internet Explorer or your preferred browser.

Microsoft Exchange 2010 – Public Folder Replicas

A major part of any Microsoft Exchange transition is the task of moving all public folder content. Sometimes Ive found this to be a migration sticking point, awaiting all of your public folder content to move from one Exchange Organisation to another. One thing I have found that speed’s up the process, are two PowerShell scripts that are supplied “out of the box”  with Microsoft  Exchange 2010. To benefit from these, perform the following actions:

1. Open the Exchange Management Shell (EMS) and change directory to:

“C:\Program Files\Microsoft\Exchange Server\V14\Scripts\”

2. The following PowerShell script will add your new Exchange 2010 server as replica on all public folders in your existing public folder database. In the same EMS window you opened in step one, run the following command. Please note, replace the “MyExchange2010Server” entry with the name of your own Exchange 2010 server.

AddReplicaToPFRecursive.ps1 -TopPublicFolder “\” -ServerToAdd “MyExchange2010Server”

3. Once this command has completed, we can then run a second PowerShell script to initiate the move of all public folder content. To perform this task, in the same EMS window run the following command. Please note, replace the “MyExchange200xServer” and “MyExchange2010Server” entries with the names of your own legacy and  Exchange 2010 servers.

MoveAllReplicas.ps1 -Server “MyExchange200xServer” -NewServer “MyExchange2010Server”

Once this command is complete, your public folder content will now start to replicate. You can monitor it’s progress by viewing the “Public Folder Instances” container in Exchange System Manager on Microsoft Exchange 2003 or through the Public Folder Management Console in Microsoft Exchange 2007.

As a note, if you are transitioning to a Microsoft Exchange 2007 organisation, these PowerShell scripts are also available and can be found in the following directory, C:\Program Files\Microsoft\Exchange Server\Scripts.