SfB Front-End Service stuck on Starting due to Cert issue

Hi All,

the argument “SfB Front-End Service stuck on Starting” is one of the most common query for an SfB Admin.
There are many different reasons for this issue, here I document one of them that I found quite often.

Lync / SfB Server Fron-End stuck on Starting

One of the Event Log error you get is this one

Source: LS User Services
Event ID: 32174
Description: Server startup is being delayed because fabric pool manager has not finished initial placement of users.
Currently waiting for routing group: {A43BBA3A-8963-51C4-BD7A-19E1EC3DDFDB}.
Number of groups potentially not yet placed: 7.
Total number of groups: 7.
Cause: This is normal during cold-start of a Pool and during server startup.
If you continue to see this message many times, it indicates that insufficient number of Front-Ends are available in the Pool.
Resolution:
During a cold-start of a large Pool it can take upto an hour for the placement process to finish as it needs to populate all the Front-End databases with data from the Backup Store. If the Pool is running and the Front-End is just started, this is normal for some time. If this repeats for a long time, ensure that all the Front-Ends configured for this Pool are up and running. If multiple Front-Ends have been recently decommissioned, run Reset-CsPoolRegistrarState -ResetType QuorumLossRecovery to enable the Pool to recover from Quorum Loss and make progress.

If you have a Front-End Service stuck on Starting after a Reboot, try immediately this PowerShell command:

Get-Childitem cert:\LocalMachine\root -Recurse | Where-Object {$_.Issuer -ne $_.Subject} | Format-List *

this PS Command compares the “Issuer” property and the “Subject” property of each certificate in the store, and then outputs details of certificates that do not meet the criteria of a self-signed certificate.

This issue occurs because a certificate that is not self-signed was installed in the Trusted Root Certification Authorities store.
This is an incorrect configuration that can cause HTTP communication between Lync/SfB servers to fail with an untrusted root certificate error. Lync/SfB deployments in Windows Server 2012 and above may experience this issue because Windows Server 2012 and above implements checks for a higher level of trust for certificate authentication.

Note: A self-signed certificate is defined as a certificate in which the “Issuer” property and the “Subject” property on the certificate are the same. This is normal and expected for Root Certification Authorities.

Recently we have found this two Intermediate Certificates preventing Lync/SfB Front-End Service to Start

Get-Childitem cert:\LocalMachine\root -Recurse | Where-Object {$_.Issuer -ne $_.Subject} | Format-List *
PSPath : Microsoft.PowerShell.Security\Certificate::LocalMachine\root\92C1588E85AF2201CE7915E8538B492F605B80C6
PSParentPath : Microsoft.PowerShell.Security\Certificate::LocalMachine\root
PSChildName : 92C1588E85AF2201CE7915E8538B492F605B80C6
PSDrive : Cert
PSProvider : Microsoft.PowerShell.Security\Certificate
PSIsContainer : False
EnhancedKeyUsageList : {Code Signing (1.3.6.1.5.5.7.3.3)}
DnsNameList : {DigiCert SHA2 Assured ID Code Signing CA}
SendAsTrustedIssuer : False
EnrollmentPolicyEndPoint : Microsoft.CertificateServices.Commands.EnrollmentEndPointProperty
EnrollmentServerEndPoint : Microsoft.CertificateServices.Commands.EnrollmentEndPointProperty
PolicyId :
Archived : False
Extensions : {System.Security.Cryptography.Oid,System.Security.Cryptography.Oid,System.Security.Cryptography.Oid,System.Security.Cryptography.Oid…}
FriendlyName :
IssuerName : System.Security.Cryptography.X509Certificates.X500DistinguishedName
NotAfter : 22/10/2028 14.00.00
NotBefore : 22/10/2013 14.00.00
HasPrivateKey : False
PrivateKey :
PublicKey : System.Security.Cryptography.X509Certificates.PublicKey
RawData : {48, 130, 5, 48…}
SerialNumber : 0409181B5FD5BB66755343B56F955008
SubjectName : System.Security.Cryptography.X509Certificates.X500DistinguishedName
SignatureAlgorithm : System.Security.Cryptography.Oid
Thumbprint : 92C1588E85AF2201CE7915E8538B492F605B80C6
Version : 3
Handle : 1016668239072
Issuer : CN=DigiCert Assured ID Root CA,OU=www.digicert.com, O=DigiCert Inc, C=US
Subject : CN=DigiCert SHA2 Assured ID Code Signing CA,OU=www.digicert.com, O=DigiCert Inc, C=US
PSPath : Microsoft.PowerShell.Security\Certificate::LocalMachine\root\495847A93187CFB8C71F840CB7B41497AD95C64F
PSParentPath : Microsoft.PowerShell.Security\Certificate::LocalMachine\root
PSChildName : 495847A93187CFB8C71F840CB7B41497AD95C64F
PSDrive : Cert
PSProvider : Microsoft.PowerShell.Security\Certificate
PSIsContainer : False
EnhancedKeyUsageList : {Client Authentication (1.3.6.1.5.5.7.3.2), CodeSigning (1.3.6.1.5.5.7.3.3)}
DnsNameList : {VeriSign Class 3 Code Signing 2010 CA}
SendAsTrustedIssuer : False
EnrollmentPolicyEndPoint : Microsoft.CertificateServices.Commands.EnrollmentEndPointProperty
EnrollmentServerEndPoint : Microsoft.CertificateServices.Commands.EnrollmentEndPointProperty
PolicyId :
Archived : False
Extensions : {System.Security.Cryptography.Oid,System.Security.Cryptography.Oid,System.Security.Cryptography.Oid,System.Security.Cryptography.Oid…}
FriendlyName :
IssuerName : System.Security.Cryptography.X509Certificates.X500DistinguishedName
NotAfter : 08/02/2020 00.59.59
NotBefore : 08/02/2010 01.00.00
HasPrivateKey : False
PrivateKey :
PublicKey : System.Security.Cryptography.X509Certificates.PublicKey
RawData : {48, 130, 6, 10…}
SerialNumber : 5200E5AA2556FC1A86ED96C9D44B33C7
SubjectName : System.Security.Cryptography.X509Certificates.X500DistinguishedName
SignatureAlgorithm : System.Security.Cryptography.Oid
Thumbprint : 495847A93187CFB8C71F840CB7B41497AD95C64F
Version : 3
Handle : 1016668237408
Issuer : CN=VeriSign Class 3 Public Primary CertificationAuthority - G5, OU="(c) 2006 VeriSign, Inc. - For authorized use only", OU=VeriSign Trust Network,O="VeriSign, Inc.", C=US
Subject : CN=VeriSign Class 3 Code Signing 2010 CA, OU=Terms of use at https://www.verisign.com/rpa (c)10,OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US

These two valid certificates are misplaced in the Trusted Root Certification Authorities Store and are also, correctly, placed in the Intermediate Certification Authorities Store

To solve the issue, delete this two certificates from the
Trusted Root Certification Authorities Store, but leave them in the Intermediate Store

To be sure, after you have deleted these certs, execute again the PowerShell command, you should get nothing this time.

Reboot the Server, after that the Front-End Service should start correctly.

I hope this could help some of you.
Regards
Luca

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Create a free website or blog at WordPress.com.

Up ↑

%d bloggers like this: