CODEWITHSUNDEEP
architectureautoscalingcomplex-event-processingstreaminsight
Paul Fryer
Paul Fryer

Reputation: 9527

Approaches for scaling out StreamInsight?

I am considering using StreamInsight for an CEP system I'm working on, however the scale out story doesn't seem to exist in any of the documentation I could find online. StreamInsight seems to be designed to run on a single server. The size of the system I'm looking at could reach tens of thousands of transactions per second and needs to have a high SLA. So I need to understand how (if anyone has) scaled StreamInsight deployments horizontally, meaning over multiple servers to achieve high availability.

It looks like the SI team was thinking of this in 2011, but its not clear anything has come from that since the latest build of SI is from 2012.

Upvotes: 3

Views: 256

Answers (1)

DevBiker
DevBiker

Reputation: 451

There is no official "scale out" story for StreamInsight. However, one solution would be to have multiple servers running the queries and then, based on a parameter, only a single server actually "outputting" the results. It would be pretty complex but certainly doable. For resiliency, you can use checkpointing ... this will write the state of the event engine to disk so that you can recover the running queries. Depending on the queries that you have, this may or may not be helpful. One comment - "tens of thousands" of events/second is something that can easily be done on a single server.

Upvotes: 4

Related Questions