CODEWITHSUNDEEP
kdb+
Utsav
Utsav

Reputation: 5918

Can slow real time subscriber kill tickerplant in kdb

Can a slow consumer kill or slow down tickerplant?

I have a ticker plant which has 3 realtime subscribers of which one subscriber is slow.

q).z.W
7 | `long$()
8 | 969393 198 198 197 197 198 196 199 197 196 143 198 196 196 197 197 198 19..
9 | 199 198 198 143 197 199 197 197 197 197 199 196 199 145 196 198 198 198 1..
10| 198 196 198 144 199 198 198 198 196 197 196 199 198 143 199 198 197 198 1..

q)count each .z.W
7 | 0
8 | 85547
9 | 77931
10| 0

q)count each .z.W
7 | 0
8 | 191552
9 | 0
10| 0

Can the slow consumer get tickerplant killed or slow it down in production kdb+ systems receiving billions of records?

Upvotes: 0

Views: 1017

Answers (2)

Callum Biggs
Callum Biggs

Reputation: 1540

Terrys answer is 100% correct. I just want to expand on the need for garbage collection in a TP with slow subscribers.

It's important that this collection is implemented via .Q.gc[] not by immediate collection \g 1. The immediate collection is only triggered when all references to an object are dropped and the object is returned to heap, if it is greater than 64MB then collection triggers.

During normal execution in a TP, the published data is never a referenced object, it is the parameters of an incoming message that is then published out. Because of this, there is no object de-referencing that triggers the automatic garbage collection.

Upvotes: 3

terrylynch
terrylynch

Reputation: 13572

Yes, a slow consumer can kill a tickerplant. The slow consumer creates the output queue in the tickerplant and this output queue consumes memory. Eventually if it goes on long enough the tickerplant (or the machine it's running on) can run out of memory and abort.

Ideally a production tickerplant will have some form of monitoring which periodically keeps an eye on the output queues - if a queue gets beyond a certain threshold it should halt the subscription (temporarily remove the handle from the .u.w subscription dictionary, allow the queue to drain) and resume if/when the subscriber catches up. Or be more aggressive and close the subscribers connection entirely (hclose) which wipes the output queue.

If your system experiences a lot of queueing then the tickerplant might also need a daily garbage collect too (say at EOD) to make sure that output queues haven't caused it to hold unused memory (or maybe you want to keep it with unused memory so that it doesn't have to re-request memory from the OS next time there's a big queue)

Upvotes: 5

Related Questions