CODEWITHSUNDEEP
azure-service-fabric
Sean Feldman
Sean Feldman

Reputation: 26057

Deploying number of stateless services larger than number of cluster nodes

I've noticed that when deploying stateless service to a cluster, the maximum number of instances cannot exceed the number of physical nodes. Two questions about that:

  1. Is that a hard limit by design (i.e. not just a dev cluster issue)?
  2. What happens to that service`s processing ability when half a cluster is down?

Upvotes: 2

Views: 1161

Answers (1)

yoape
yoape

Reputation: 3335

The number of instances can not exceed the number of nodes in the cluster, no, there is a placement constraint that ensures that. You can modify placement constraints for your cluster, but I don't think you can set it to ignore the ReplicaExclusionStatic constraint completely. It's also worth to note that in the default setting it is treated as a warning, i.e. if you have node count = 7 and your cluster has 5 nodes, it will give you warning that it is unable to place 2 instances.

In Service Fabric there is both the concept of instance and partition, and most commonly instance is associated with Stateless services, and partition with Stateful, but that doesn't mean that you cannot have partitions of Stateless services (but it might not make sense in most cases either).

If you wish to have more actual instances running of your service then you could change the partition count in order to create further partitions on each node. The default for a Stateless service is SingletonPartition and an instance count of -1 that puts one instance on each available node:

In ApplicationManifest.xml:

<Service Name="MyService">
    <StatelessService ServiceTypeName="MyServiceType" InstanceCount="[MyService_InstanceCount]">
        <SingletonPartition />
    </StatelessService>
</Service>

And in your PublishProfiles/Cloud.xml:

<Parameters>
    <Parameter Name="MyService_InstanceCount" Value="-1" />
</Parameters>

On a 5 node cluster you will get 5 instances running, nothing strange there. If you change to instance count = 6 then you will get the placement warning (and still no more than 5 actual instances running).

Now, if you want to have more instances (or replicas) running on each node you could modify the partition count of your Stateless service:

<Service Name="MyService">
    <StatelessService ServiceTypeName="MyServiceType" InstanceCount="[MyService_InstanceCount]">
        <UniformInt64Partition PartitionCount="4" LowKey="0" HighKey="4" />
    </StatelessService>
</Service>

On the same cluster and with the same instance count you will instead get 20 (5 nodes x 4 partitions) replicas of your service running now.

What complicates it now is that you will always have to include a partitionkey when you talk to your Stateless services. It might not make sense in the same way it does for a Stateful service as there is no expected state, but your selection of partition key should be evenly distributed to even the load.

When you then scale your cluster you will always get the same number of replicas per partitions, but the number of partitions per node may differ if you set instance count to a specific number instead of -1.

This article discusses more about partitioning, instances and replicas. https://blogs.msdn.microsoft.com/mvpawardprogram/2015/10/13/understanding-service-fabric-partitions/

Upvotes: 4

Related Questions