CODEWITHSUNDEEP
azure-service-fabric
Per
Per

Reputation: 521

Service Fabric Error: Failed to ACL folders or certificates required by application. Error:FABRIC_E_CERTIFICATE_NOT_FOUND

I am stumped on this one. I have a secure cluster with some encrypted application settings. The app runs fine on my local cluster, but not when deployed to the cloud.

The application deploys ok, but fails to start up with the following error: Failed to ACL folders or certificates required by application. Error:FABRIC_E_CERTIFICATE_NOT_FOUND.

I have created a self signed cert, exported it (with the private key) to a PFX, and uploaded it to the vault:

New-SelfSignedCertificate -Type DocumentEncryptionCert -KeyUsage     DataEncipherment -Subject mycert -Provider 'Microsoft Enhanced Cryptographic Provider v1.0'

Invoke-AddCertToKeyVault -SubscriptionId 'xxxxx-bxxxxfb9-xxxx-xxx-xxxxx' -ResourceGroupName 'vault-sec-studio-dev' -Location "Central US" -VaultName 'vault-sec-studio-dev' -CertificateName 'mycert' -Password "myPass" -UseExistingCertificate -ExistingPfxFilePath "C:\temp\Azure\Dev\mycert.pfx"

I add the certificate to the cluster security tab by referencing the thumbprint: enter image description here

enter image description here

I update the ApplicationManifest.xml:

<Principals>
    <Users>
      <User Name="Service1" AccountType="NetworkService" />
    </Users>
  </Principals>
  <Policies>
    <SecurityAccessPolicies>
      <SecurityAccessPolicy GrantRights="Read" PrincipalRef="Service1" ResourceRef="mycert" ResourceType="Certificate"/>
    </SecurityAccessPolicies>
  </Policies>
  <Certificates>
    <SecretsCertificate X509FindValue="72C57495F3034E072CA6F536EEABE984AA869CBC" X509StoreName="My"   X509FindType="FindByThumbprint" Name="mycert"  />
  </Certificates>

The Sevice Fabric Explorer page shows the upgrade was installed but failed to start. Several nodes are in error status:

enter image description here When remoting in to the VM's I don't see this cert installed. I see the main cert used to secure the cluster, but not this admin cert.

I have tried to manually install the cert on each VM, but getting the same result.

I have spent a ton of time on this, and can't seem to get anywhere, so I'm hopeful someone can give me some pointers here.

Update: I'm seeing this in the event log on the VM. Talks about the private key which makes me think there is something wrong with the cert, or pfx?

Failed to get the Certificate's private key. Thumbprint:72C57495F3034E072CA6F536EEABE984AA869CBC. Error: FABRIC_E_CERTIFICATE_NOT_FOUND

Upvotes: 2

Views: 4970

Answers (1)

Per
Per

Reputation: 521

This issue is now finally resolved. I can't say I completely understand this, but this is what I found:

  1. Don't create the Service Fabric Cluster using the Portal. You'll need to use the template so you have access to configure the certs.

  2. Also no need to mess around with Admin Certs on the security tab that I originally did (see original question). Those don't work for this, or at least not the way I'd expect them to.

  3. You must edit the ARM template and add the following certificate information to the "secrets" array on each VM:

enter image description here

Note section above is added and points to a Certificate URL

"settingCertificateUrlValue": { "value": "https://vault-my-site.vault.azure.net:443/secrets/studiosecrets/487a94749ee148979cc97a68abe9fd3a" },

The cluster is now "green" and the app runs fine.

Upvotes: 4

Related Questions