CODEWITHSUNDEEP
c#.netcomplex-event-processingstreaminsight
fred
fred

Reputation: 131

StreamInsight and long running queries

I have an application built on top of StreamInsight 1.2 that has several standing queries running some calculations in User Defined Operators. Occasionally one of the calculations will take a relatively long time to complete (a few seconds), and I'm seeing that it blocks the other queries from executing. I'm not very familiar with how StreamInsight works so from this I'm speculating that all queries are run on a single thread. Is there some way to have the queries run in parallel?

Aside from somehow putting the queries on separate threads, are there any other things I could do to prevent one sporadically long running query from blocking other queries?

Upvotes: 0

Views: 76

Answers (2)

DevBiker
DevBiker

Reputation: 451

If the calculation will occasionally take a few seconds and that's the nature of it and you are running Standard Edition, the best thing that you can do is to not block the thread while the calculation is taking place. StreamInsight doesn't have anything built-in for this ... it's your responsibility. From StreamInsight's perspective, your UDO is taking some time. It doesn't know why that is. And if the thread is blocked, it can't run anything else until your UDO returns. So I would a) validate that you are running standard edition and then b) if you are, change the UDO so that it's non-blocking/async.

Upvotes: 1

DevBiker
DevBiker

Reputation: 451

Which edition of StreamInsight are you using? Standard Edition runs the scheduler on a single core while Premium uses all available cores. As for the delay, if all of your queries are dependent on the timeline that's hung up - via join, union, etc. - then that's what's causing it to hang up. Events aren't "released" from the long-running UDO to feed the other streams. If this is the case, there isn't anything that you can do unless you can restructure the queries in some way. I'd also start looking into why "occasionally" the UDO will take a few seconds. This leads me to think that there's something amiss with the UDO but then, not knowing the details of what it's doing, I can't say for sure.

Upvotes: 1

Related Questions