CODEWITHSUNDEEP
multithreadinggoconcurrencylocking
ambikanair
ambikanair

Reputation: 4600

Which data type to use in Go for concurrent read/write operation

I have a scenario where I create a worker pool which has 4 workers. These workers picks up job from a channel and stores the result to the memory. So the worker execute the job and may update the result in parallel. I would like to know which data type is best to save the result as it would be accessed by multiple workers at a time. I would need to use the lock mechanism to achieve this.

The result will be key-value format. So can maps be considered in this scenario.

Upvotes: 2

Views: 655

Answers (2)

user9455968
user9455968

Reputation:

I'd use a second channel the workers send results to. One goroutine will receive results from this channel and store them. Shared memory and protected code sections are not needed if you use this approach.

Edit: Example

The following program uses four workers and one result collector. Tasks are passed to the workers from the main gorouting. The worker send results and errors to the collector who builds a summary.

package main

import (
    "fmt"
    "sync"
)

func main() {
    tasks := make(chan int)
    results := make(chan string)

    var wg sync.WaitGroup
    for i := 0; i < 4; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for task := range tasks {
                rem := int(task) % 2
                switch rem {
                case 0:
                    results <- fmt.Sprintf("done with task %d", task)
                case 1:
                    results <- fmt.Sprintf("error: only even tasks accepted: %d", task)
                }
            }
        }()
    }

    summary := ""
    go func() {
        for result := range results {
            summary += result + "\n"
        }
    }()

    for i := 0; i < 10; i++ {
        tasks <- i
    }

    close(tasks)
    wg.Wait()
    close(results)

    fmt.Println(summary)
}

see https://play.golang.org/p/puZONnJHme-

Upvotes: 2

TehSphinX
TehSphinX

Reputation: 7440

Yes, you could use a map for that and you would have to protect it by a mutex.

I would however suggest that each job contains a result channel where you can input the result of the job. Here a rough sketch of what that would look like:

playground: note that playground only has one core so the numbers will appear in sequence.

package main

import (
    "fmt"
    "time"
)

type job struct {
    a     int
    b     int
    chRes chan int
}

func main() {

    var chIn = make(chan job, 20)

    for i := 0; i < 10; i++ {
        go worker(chIn)
    }

    for i := 0; i < 100; i++ {
        go func(i int) {
            chRes := make(chan int)
            chIn <- job{
                a:     i,
                b:     i,
                chRes: chRes,
            }

            fmt.Println(<-chRes)
        }(i)
    }

    // I'm lazy, so here just a sleep so we can see something
    time.Sleep(1 * time.Second)
}

func worker(ch chan job) {
    for job := range ch {
        job.chRes <- job.a + job.b
    }
}

-- edit --

If you device to go with a store instead (which kind of goes against the go mantra share memory by communicating instead of communicate by sharing memory): there is also a sync.Map that is already concurrency safe. No mutex needed.

Upvotes: 1

Related Questions