Fatal error: goroutines are asleep - deadlock

Question

Trying to learn go concurrency. I was having an issue where I was getting an error of:

fatal error: all goroutines are asleep - deadlock!

And I was told to add a waitgroup and a close channel to fix that issue. I've added both of those, but the error still persists. Not sure what I'm doing wrong anymore.

Here's my code https://play.golang.org/p/ZB45oXlBUl:

package main

import (
    "log"
    "sync"
    "time"
)

type RowInfo struct {
    id int64
}

func main() {
    queueChan := make(chan RowInfo)
    workerChan := make(chan RowInfo)
    doneChan := make(chan int64)
    closeChan := make(chan struct{})

    var waitGroup sync.WaitGroup

    go dispatcher(queueChan, workerChan, doneChan, closeChan)

    // Start WorkerCount number of workers
    workerCount := 4
    for i := 0; i < workerCount; i++ {
        go worker(workerChan, doneChan, &waitGroup)
    }

    // Send test data
    waitGroup.Add(12)
    for i := 0; i < 12; i++ {
        queueChan <- RowInfo{id: int64(i)}
    }

    // Prevent app close till finished execution
    waitGroup.Wait()

    close(closeChan)
}

func dispatcher(queueChan, workerChan chan RowInfo, doneChan chan int64, closeChan chan struct{}) {
    state := make(map[int64]bool)

    for {
        select {
        case job := <-queueChan:
            if state[job.id] == true {
                continue
            }
            workerChan <- job
        case result := <-doneChan:
            state[result] = false
        case <-closeChan:
            close(queueChan)
            close(workerChan)
            close(doneChan)
            break
        }
    }
}

func worker(workerChan chan RowInfo, doneChan chan int64, waitGroup *sync.WaitGroup) {
    for job := range workerChan {
        time.Sleep(1 * time.Second)
        log.Printf("Doing work on job rowInfo ID: %d", job.id)

        // Finish job
        doneChan <- job.id
        waitGroup.Done()
    }
}

And the errors:

2009/11/10 23:00:01 Doing work on job rowInfo ID: 2
2009/11/10 23:00:01 Doing work on job rowInfo ID: 0
2009/11/10 23:00:01 Doing work on job rowInfo ID: 3
2009/11/10 23:00:01 Doing work on job rowInfo ID: 1
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan send]:
main.main()
    /tmp/sandbox490337982/main.go:32 +0x1e0

goroutine 5 [chan send]:
main.dispatcher(0x104360c0, 0x10436100, 0x10436140, 0x10436180)
    /tmp/sandbox490337982/main.go:50 +0x200
created by main.main
    /tmp/sandbox490337982/main.go:21 +0x100

goroutine 6 [chan send]:
main.worker(0x10436100, 0x10436140, 0x104382e0, 0x0)
    /tmp/sandbox490337982/main.go:68 +0x1a0
created by main.main
    /tmp/sandbox490337982/main.go:26 +0x160

goroutine 7 [chan send]:
main.worker(0x10436100, 0x10436140, 0x104382e0, 0x0)
    /tmp/sandbox490337982/main.go:68 +0x1a0
created by main.main
    /tmp/sandbox490337982/main.go:26 +0x160

goroutine 8 [chan send]:
main.worker(0x10436100, 0x10436140, 0x104382e0, 0x0)
    /tmp/sandbox490337982/main.go:68 +0x1a0
created by main.main
    /tmp/sandbox490337982/main.go:26 +0x160

goroutine 9 [chan send]:
main.worker(0x10436100, 0x10436140, 0x104382e0, 0x0)
    /tmp/sandbox490337982/main.go:68 +0x1a0
created by main.main
    /tmp/sandbox490337982/main.go:26 +0x160

Answer 1

In order to understand the issue, think about what is going on with the dispatcher and the workers.

1. Initial state: dispatcher and workers are idle
2. Send on queueChan: dispatcher select sends to workerChan
3. workerChan read by worker: sleep 4 seconds, write to doneChan.
4. 2, 3 is repeated 4 times until all workers are sleeping
5. While all workers are sleeping, another job comes in on queueChan.
6. dispatcher goes to do that work
7. dispatcher can not send on workerChan because no workers are reading.
8. all workers get through their sleep and try to send on doneChan.

Now all goroutines are blocked.