CODEWITHSUNDEEP
azure-service-fabric
Jerome Haltom
Jerome Haltom

Reputation: 1767

Publishing metadata to Service Fabric

So, I have this idea I'm working on, where services on some nodes need to discover other services dynamically at runtime, based on metadata that they might publish. And I'm trying to figure out the best way to go about this.

Some of this metadata will be discovered from the local machine at runtime, but it then has to be published to the Fabric so that other services can make decisions on it.

I see the Extension stuff in the ServiceManifests. This is a good start. But it doesn't seem like you can alter or add extensions at runtime. That would be nice!

Imagine my use case. I have a lot of machines on a Fabric, with a lot of services deployed to them. What I'm advertising is the audio codecs that a given machine might support. Some nodes have DirectShow. So, they would publish the local codecs available. Some machines are running 32 bit services, and publish the 32 bit DirectShow codecs they have (this is actually what I need, since I have some proprietary ACM codecs that only run in 32 bit). Some machines are Linux machines, and want to make available their GStreamer codecs.

Each of these needs to publish the associated metadata about what they can do, so that other services can string together from that metadata a graph about how to process a given media file.

And then each will nicely report their health and load information, so the fabric can determine how to scale.

Each of these services would support the same IService interface, but each would only be used by clients that decided to use them based on the published metadata.

Upvotes: 2

Views: 311

Answers (1)

Vaclav Turecek
Vaclav Turecek

Reputation: 9050

In Service Fabric the way to think about this kind of problem is from a service point of view, rather than a machine point of view. In other words, what does each service in the cluster support, rather than what does each machine support. This way you can use a lot of Service Fabric's built-in service discovery and querying stuff, because the abstraction the platform provides is really about services more than it is about machines.

One way you can do this is with placement constraints and service instances representing each codec that the cluster supports as a whole. What that means is that you'll have an instance of a service representing a codec that only runs on machines that support that codec. Here's a simplified example:

Let's say I have a Service Type called "AudioProcessor" which does some audio processing using whatever codec is available.

And let's I have 5 nodes in the cluster, where each node supports one of codecs A, B, C, D, and E. I will mark each node with a node property corresponding to the codec it supports (a node property can just be any string I want). Note this assumes I, the owner of the cluster, know the codecs supported by each machine.

Now I can create 5 instances of the AudioProcessor Service Type, one for each codec. Because each instance gets a unique service name that is in URI format, I can create a hierarchy with the codec names in it for discovery through Service Fabric's built-in Naming Service and querying tools, e.g., "fabric:/AudioApp/Processor/A" for codec A. Then I use a placement constraint for each instance that corresponds to the node property I set on each node to ensure the codec represented by the service instance is available on the node.

Here's what all this looks like when everything is deployed:

Node 1 - Codec: A Instance: fabric/AudioApp/Processor/A

Node 2 - Codec: B Instance: fabric/AudioApp/Processor/B

Node 3 - Codec: C Instance: fabric/AudioApp/Processor/C

Node 4 - Codec: D Instance: fabric/AudioApp/Processor/D

Node 5 - Codec: E Instance: fabric/AudioApp/Processor/E

So now I can do things like:

  • Find all the codecs the cluster supports by querying for a list of AudioProcessor service instances and examining their names (similar to getting a list of URIs in an HTTP API).
  • Send a processing request to the service that supports codec B by resolving fabric:/AudioApp/AudioProcessor/B
  • Scale out processing capacity of codec C by adding more machines that support codec C - Service Fabric will automatically put a new "C" AudioProcessor instance on the new node.
  • Add machines that support multiple codecs. Using multiple node properties on it, Service Fabric will automatically place the correct service instances on it.

The way a consumer thinks about this application now is along the lines of "is there a service that support codec E?" or "I need to talk to service A, C, and D to process this file because they have the codecs I need."

Upvotes: 4

Related Questions