Dynamic Servicefabric Settings and Overrides

Question

Is there a way to not tell the service about the settings at all and just provide them at the application level?

I am still unhappy with how servicefabric configuration seems to work.

Near as I can tell I have to specify in the service’s settings.xml all of the possible configuration values. Then I can override those in the application’s ApplicationParameters. Per documentation this looks like it holds true for environment variables also.

The complication that creates is that our configuration is used to hydrate options in many cases with arrays.

For example consider the json:

{
  "AuthorizationOptions": {
    "Policies": [
      {
        "Name": "User",
        "Groups": [ "Domain Users" ]
      }
    ]
  }  
}

There are 2 arrays; that are necessary and useful. To express this in the service fabric configuration it translates to:

<Section Name="AuthorizationOptions">
    <Parameter Name="Policies:0:Name" Value="User"/>
    <Parameter Name="Policies:0:Groups:0" Value="Domain Users"/>
</Section>

While the translation is not pleasant in comparison to the structured object it is completely usable.

However, If I don’t specify the section and parameters in the service, I can’t seem to override them in the application. So in this case I would have to define the exact number of policies and groups per policy in the service and the application could modify the policy name, or the group values, but not the number of policies total or number of groups total.

Is there a way to not tell the service about the settings at all and just provide them at the application level?

If not what alternatives exist to make the service reusable across applications that I may want to use to provide this type of dynamic configuration differently?

The last part of the puzzle that may assist in answering this question is I am using some pre-release code to translate the service fabric settings into Microsoft.Extensions.Configuration.IConfiguration. However, that is just taking the settings it finds; it isn't the cause of the override issue I am running into.

Example Service Settings.xml:

<?xml version="1.0" encoding="utf-8" ?>
<Settings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/2011/01/fabric">
  <Section Name="AuthorizationOptions">
    <!-- I should not have to provide these at the application level!
         However, it fails to deploy if I don't. -->
    <Parameter Name="Policies:0:Name" Value="User"/>
    <Parameter Name="Policies:0:Groups:0" Value="Domain Users"/>
  </Section>
</Settings>

Example Application ApplicationManifest.xml:

<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ApplicationTypeName="ServiceFabric.ExampleType" ApplicationTypeVersion="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric">
  <Parameters>
    <Parameter Name="Service.Example_ASPNETCORE_ENVIRONMENT" DefaultValue="" />
    <Parameter Name="Service.Example_InstanceCount" DefaultValue="-1" />
    <Parameter Name="Service.Example_AuthorizationOptions_Policies_0_Name" DefaultValue="Users" />
    <Parameter Name="Service.Example_AuthorizationOptions_Policies_0_Groups_0" DefaultValue="Domain Users" />
  </Parameters>
  <ServiceManifestImport>
  <ServiceManifestImport>
    <ServiceManifestRef ServiceManifestName="Service.ExamplePkg" ServiceManifestVersion="1.0.0" />
    <ConfigOverrides>
      <ConfigOverride Name="Config">
        <Settings>
          <Section Name="AuthorizationOptions">
            <Parameter Name="Policies:0:Name" Value="[Service.Example_AuthorizationOptions_Policies_0_Name]" />
            <Parameter Name="Policies:0:Groups:0" Value="[Service.Example_AuthorizationOptions_Policies_0_Groups_0]" />
          </Section>
        </Settings>
      </ConfigOverride>
    </ConfigOverrides>
    <EnvironmentOverrides CodePackageRef="code">
      <EnvironmentVariable Name="ASPNETCORE_ENVIRONMENT" Value="[Service.Example_ASPNETCORE_ENVIRONMENT]" />
    </EnvironmentOverrides>
  </ServiceManifestImport>
  <DefaultServices>
    <Service Name="Service.Example" ServicePackageActivationMode="ExclusiveProcess">
      <StatelessService ServiceTypeName="Service.ExampleType" InstanceCount="[Service.Example_InstanceCount]">
        <SingletonPartition />
      </StatelessService>
    </Service>
  </DefaultServices>
</ApplicationManifest>

Example Application ApplicationParameters (Local.1Node.xml):

<?xml version="1.0" encoding="utf-8"?>
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/ServiceFabric.Example" xmlns="http://schemas.microsoft.com/2011/01/fabric">
  <Parameters>
    <Parameter Name="Service.Example_ASPNETCORE_ENVIRONMENT" Value="Development" />
    <Parameter Name="Service.Example_InstanceCount" Value="-1" />
    <Parameter Name="Service.Example_AuthorizationOptions_Policies_0_Name" Value="Users" />
    <Parameter Name="Service.Example_AuthorizationOptions_Policies_0_Groups_0" Value="Domain Users" />
  </Parameters>
</Application>

Example Application PublishProfiles (Local.1Node.xml):

<?xml version="1.0" encoding="utf-8"?>
<PublishProfile xmlns="http://schemas.microsoft.com/2015/05/fabrictools">
  <ClusterConnectionParameters />
  <ApplicationParameterFile Path="..\ApplicationParameters\Local.1Node.xml" />
</PublishProfile>

Should be unrelated, but example consumption of the settings:

internal sealed class Example : StatelessService
{
    public Example(StatelessServiceContext context)
        : base(context)
    { }

    protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
    {
        return new ServiceInstanceListener[]
        {
            new ServiceInstanceListener(serviceContext =>
                new HttpSysCommunicationListener(serviceContext, "ServiceEndpoint", (url, listener) =>
                {
                    ServiceEventSource.Current.ServiceMessage(serviceContext, $"Starting HttpSys on {url}");

                    return new WebHostBuilder()
                            .UseHttpSys(options =>
                            {
                                options.Authentication.Schemes = AuthenticationSchemes.Negotiate; // Microsoft.AspNetCore.Server.HttpSys
                                                                                        options.Authentication.AllowAnonymous = false;
                            }).ConfigureServices(services => services.AddSingleton<StatelessServiceContext>(serviceContext))
                                        .UseContentRoot(Directory.GetCurrentDirectory())
                                        .ConfigureAppConfiguration((hostingContext, config) =>
                                            {
                                                config.SetBasePath(Directory.GetCurrentDirectory());
                                                config.AddServiceFabricConfiguration(FabricRuntime.GetActivationContext(), options => {
                                                    options.IncludePackageName=false;
                                                });
                                            })
                                        .UseStartup<Startup>()
                                        .UseServiceFabricIntegration(listener, ServiceFabricIntegrationOptions.None)
                                        .UseUrls(url)
                                        .Build();
                }))
        };
    }
}

From that point forward everything is in the IConfiguration object as expected.

Answer 1

There are many ways to configure a service fabric application, and each approach will bring you to a different challenges.

SF team recommend the approach in the docs, because you can have a better version control of configurations, and makes harder to commit mistakes, as it is explicitly declared in a file, I've used a few different approaches because of limitations like yours, the follwoing approach might solve your problem:

Configure like the original approach, but with complex types stored as JSON values: it is the closest solution to the recommended design and you still can keep control of the configuration versions on source control.

It would be something like:

Settings.xml:

<?xml version="1.0" encoding="utf-8" ?>
<Settings xmlns... namespaces here...>
  <Section Name="AuthorizationOptions">
    <Parameter Name="Policies"/>
  </Section>
</Settings>

ApplicationManifest.xml:

<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest ApplicationTypeName="ServiceFabric.ExampleType" ApplicationTypeVersion="1.0.0" xmlns:...namespaces....>
  <Parameters>
    <Parameter Name="Service.Example_ASPNETCORE_ENVIRONMENT" DefaultValue="" />
    <Parameter Name="Service.Example_InstanceCount" DefaultValue="-1" />
    <Parameter Name="Service.Example_AuthorizationOptions_Policies" DefaultValue="[]" />
  </Parameters>
  <ServiceManifestImport>
  <ServiceManifestImport>
    <ServiceManifestRef ServiceManifestName="Service.ExamplePkg" ServiceManifestVersion="1.0.0" />
    <ConfigOverrides>
      <ConfigOverride Name="Config">
        <Settings>
          <Section Name="AuthorizationOptions">
            <Parameter Name="Policies" Value="[Service.Example_AuthorizationOptions_Policies]" />
          </Section>
        </Settings>
      </ConfigOverride>
    </ConfigOverrides>
    <EnvironmentOverrides CodePackageRef="code">
      <EnvironmentVariable Name="ASPNETCORE_ENVIRONMENT" Value="[Service.Example_ASPNETCORE_ENVIRONMENT]" />
    </EnvironmentOverrides>
  </ServiceManifestImport>
  <DefaultServices>
    <Service Name="Service.Example" ServicePackageActivationMode="ExclusiveProcess">
      <StatelessService ServiceTypeName="Service.ExampleType" InstanceCount="[Service.Example_InstanceCount]">
        <SingletonPartition />
      </StatelessService>
    </Service>
  </DefaultServices>
</ApplicationManifest>

ApplicationParameters.xml

<?xml version="1.0" encoding="utf-8"?>
<Application Name="fabric:/ServiceFabric.Example" xmlns:...namespaces....>
  <Parameters>
    <Parameter Name="Service.Example_ASPNETCORE_ENVIRONMENT" Value="Development" />
    <Parameter Name="Service.Example_InstanceCount" Value="-1" />
    <Parameter Name="Service.Example_AuthorizationOptions_Policies" Value="[{'Name': 'User','Groups': ['Domain Users']}, {'Name': 'Admin','Groups': ['Administrators']}]" />
  </Parameters>
</Application>

In your service code:

public class Policy
{
    public string Name { get; set; }
    public string[] Groups { get; set; }
}


var settings = this.Context.CodePackageActivationContext.GetConfigurationPackageObject("Config").Settings;
var authOptions = settings.Sections["AuthorizationOptions"].Parameters["Policies"].Value;
var obj = JsonConvert.DeserializeObject<Policy[]>(authOptions);

You could go a level further and store the entire AuthorizationOptions as a JSON, but like said previously, more generic it become, easier will be to commit mistakes and harder to find configuration issues.