CODEWITHSUNDEEP
asynchronousgodispatchergoroutine
Mheni
Mheni

Reputation: 228

What's the ideal size of buffered channel and number of workers?

I'm trying to build an asynchronous codec. I have implemented a job dispatcher that has access to a buffered channel of jobs

var JobChannel chan Job = make(chan Job, 100000)

the dispatcher takes as input the number of workers and assigns work to them

func StartDispacher(numberOfWorkers int){
    // start workers
    wg := &sync.WaitGroup{}
    wg.Add(numberOfWorkers)
    for i := int(1); i <= numberOfWorkers; i++ {
        go func(i int) {
            defer wg.Done()
            for j := range JobChannel {
                doWork(i, j)
            }
        }(i)
    }
}

my main function starts the dispatcher and keeps giving it jobs to do (in this case 200000 jobs)

workDispatcher.StartDispacher(2*runtime.NumCPU())
for i := 0; i < 200000; i++ {
    j := workDispatcher.Job{
        BytePacket: d,
        JobType:    workDispatcher.DECODE_JOB,
    }
    workDispatcher.JobChannel <- j
}

after experimenting: turns out there are 2 factors that affect the performance of this code

Is there a standard way to find the optimal values for these parameters, and is it possible to make these values independent from the physical set-up of the machine running the code?

Upvotes: 1

Views: 3612

Answers (3)

bcmills
bcmills

Reputation: 5197

In practice, I find that there are three buffer sizes that matter: 0, 1, and “an upper bound on the total number of sends”.

0 gives synchronous behavior.

1 gives asynchronous behavior: it's useful in a select statement with a default case.

An upper bound on the total number of sends gives guaranteed-non-blocking behavior: you can send to it without a select without risking a goroutine leak.

Other numbers may provide marginally better throughput, but at scale they're still going to contend on the cache line containing the channel's internal mutex, and they'll be more likely to mask potential deadlocks and goroutine leaks.

Upvotes: 4

Mr_Pink
Mr_Pink

Reputation: 109404

You always need to measure to determine how the system will perform under load. The good news here is that you only have 2 variables, which are mostly independent, so it's fairly easy to reason about.

The number of workers determines your concurrency, so benchmark the processing to see what the optimal concurrency is. There is usually a number of concurrent processes above which the returns drop off dramatically.

The size of the channel is just like any other "buffer" in a system. A larger buffer can handle larger spikes in input, at the expense of possibly inducing larger latencies and memory usage.

Upvotes: 2

lilezek
lilezek

Reputation: 7344

The answer is no. The optimal setup will be dependent not only on the software you run in doWork (how much CPU intensive and IO that function will depend) but also on how much instructions can your hardware execute and how much IO can your system deal with.

Meaning that it could depend on what your system has or has not an SSD installed or even your bandwidth if your system performs activities involving internet access, how much physical cores your CPU(s) have, etc...

Upvotes: 0

Related Questions