CODEWITHSUNDEEP
c#.nettask-parallel-librarytpl-dataflow
Maxim Zabolotskikh
Maxim Zabolotskikh

Reputation: 3367

Conditional multiple LinkTo in TPL

I have a buffer block that links to multiple action blocks:

bufferBlock.LinkTo(consumerActionBlock1, linkOptions);
bufferBlock.LinkTo(consumerActionBlock2, linkOptions);
bufferBlock.LinkTo(consumerActionBlockN, linkOptions);

The effect is that the buffer block accumulates items and once a worker-consumer can accept more items, it gets them (consumer is set to one item at a time). Like this, I have N worker threads whereas they all work on the same queue. So far so good.

Now I have a pipeline where there are two buffer blocks, each of them has workitems of certain priority:

bufferBlockPrioHigh.LinkTo(consumerActionBlockHigh1, linkOptions);
bufferBlockPrioHigh.LinkTo(consumerActionBlockHigh2, linkOptions);
bufferBlockPrioHigh.LinkTo(consumerActionBlockHighN, linkOptions);

bufferBlockPrioLow.LinkTo(consumerActionBlockLow1, linkOptions);
bufferBlockPrioLow.LinkTo(consumerActionBlockLow2, linkOptions);
bufferBlockPrioLow.LinkTo(consumerActionBlockLowN, linkOptions);

Also good. Now I want to achieve the behavior, that if there is nothing in the low priority queue, then the high priority queue could use its workers. I added this code:

bufferBlockPrioHigh.LinkTo(consumerActionBlockLow1, linkOptions, c => bufferBlockPrioLow.Count == 0);
bufferBlockPrioHigh.LinkTo(consumerActionBlockLow2, linkOptions, c => bufferBlockPrioLow.Count == 0);
bufferBlockPrioHigh.LinkTo(consumerActionBlockLowN, linkOptions, c => bufferBlockPrioLow.Count == 0);

The idea is that high priority buffer will try to send items to its own action blocks first. If they are all busy, it will try to send them to the action blocks of the low priority buffer, given that the low priority buffer is empty at the moment.

This seems to work. But what will happen, if all LinkTo fail? I.e. no own workers are available, and all conditional LinkTos will also return false (i.e. low priority queue is itself full)? Will the whole task now fail because of the new conditional LinkTos, or the task will eventually flow further once an action block, conditional or not, is free?

In other words, I'm seeking understanding in when multiple LinkTos are evaluated and how many times.

Upvotes: 1

Views: 493

Answers (1)

Maxim Zabolotskikh
Maxim Zabolotskikh

Reputation: 3367

Well, in case somebody will find this question, my findings:

This code

bufferBlockPrioHigh.LinkTo(consumerActionBlockHigh1, linkOptions);
bufferBlockPrioHigh.LinkTo(consumerActionBlockLow1, linkOptions, c => bufferBlockPrioLow.Count == 0);

makes no sense, because the tasks will always flow into consumerActionBlockHigh1, even if the block is full. I ended up with this setup to achieve what I wanted:

bufferBlockPrioHigh.LinkTo(consumerActionBlockHigh1, linkOptions, bufferBlockPrioHigh.Count < capacity);
bufferBlockPrioHigh.LinkTo(consumerActionBlockLow1, linkOptions, c => bufferBlockPrioLow.Count == 0);
bufferBlockPrioHigh.LinkTo(consumerActionBlockHigh1, linkOptions);

Which means:

  • if bufferBlockPrioHigh is not yet full, use high prio worker at once
  • try to use other capacities: else if bufferBlockPrioLow is empty, use its worker
  • else use bufferBlockPrioHigh worker

Upvotes: 1

Related Questions