all go routines are asleep deadlock

Question

I'm in the process of learning how to do concurrency, and I've written this as its own app so that I can port it into a different project once it's working.

The project I'm adding it to will basically send in a RowInfo to a global QueueChannel, and then my workers should pick up this work and process it. If I queue two rows with the same ID, and one of them is currently processing by a worker, I'll remove the duplicate row from the queue (as you can see where I do my "continue" in the dispatcher).

This queueing/worker code will be running on a web server blocking on ListenAndServe, so I want it to always remain running and the workers to always remain actively looking for jobs. I don't want to have to close the channels (unless perhaps I ctrl+C'd the app or something). I suspect the error I'm getting has something to do with not closing channels because that's what a lot of the other threads mentioning this error seem to indicate, but I'm not sure how it relates to the code I have exactly.

Terminal error output:

[~/go/src/github.com/zzz/asynch]> go run main.go
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan send]:
main.main()
    /home/zzz/go/src/github.com/zzz/asynch/main.go:29 +0x14b

goroutine 5 [select]:
main.diszzzcher(0xc82001a120, 0xc82001a180, 0xc82001a1e0)
    /home/zzz/go/src/github.com/zzz/asynch/main.go:42 +0x21a
created by main.main
    /home/zzz/go/src/github.com/zzz/asynch/main.go:19 +0xb1

goroutine 6 [chan receive]:
main.worker(0xc82001a180, 0xc82001a1e0)
    /home/zzz/go/src/github.com/zzz/asynch/main.go:55 +0x54
created by main.main
    /home/zzz/go/src/github.com/zzz/asynch/main.go:24 +0xf7

goroutine 7 [chan receive]:
main.worker(0xc82001a180, 0xc82001a1e0)
    /home/zzz/go/src/github.com/zzz/asynch/main.go:55 +0x54
created by main.main
    /home/zzz/go/src/github.com/zzz/asynch/main.go:24 +0xf7

goroutine 8 [chan receive]:
main.worker(0xc82001a180, 0xc82001a1e0)
    /home/zzz/go/src/github.com/zzz/asynch/main.go:55 +0x54
created by main.main
    /home/zzz/go/src/github.com/zzz/asynch/main.go:24 +0xf7

goroutine 9 [chan receive]:
main.worker(0xc82001a180, 0xc82001a1e0)
    /home/zzz/go/src/github.com/zzz/asynch/main.go:55 +0x54
created by main.main
    /home/zzz/go/src/github.com/zzz/asynch/main.go:24 +0xf7
exit status 2

Code:

package main

import (
    "log"
    "time"
)

type RowInfo struct {
    id int64
}

var QueueChan chan RowInfo

func main() {
    QueueChan := make(chan RowInfo)
    workerChan := make(chan RowInfo)
    exitChan := make(chan int64)

    go dispatcher(QueueChan, workerChan, exitChan)

    // Start WorkerCount number of workers
    workerCount := 4
    for i := 0; i < workerCount; i++ {
        go worker(workerChan, exitChan)
    }

    // Send test data
    for i := 0; i < 12; i++ {
        QueueChan <- RowInfo{id: int64(i)}
    }

    // Prevent app close
    for {
        time.Sleep(1 * time.Second)
    }
}

func dispatcher(queueChan, workerChan chan RowInfo, exitChan chan int64) {
    state := make(map[int64]bool)

    for {
        select {
        case job := <-QueueChan:
            if state[job.id] == true {
                continue
            }
            workerChan <- job
        case result := <-exitChan:
            state[result] = false
        }
    }
}

func worker(workerChan chan RowInfo, exitChan chan int64) {
    for job := range workerChan {
        log.Printf("Doing work on job rowInfo ID: %d", job.id)

        // Finish job
        exitChan <- job.id
    }
}

Thank you.

Answer 1

The error tells you: all goroutines are asleep, the program has deadlocked.

Now why are all your goroutines asleep? Let's check them one by one:

• the worker goroutines: Wait indefinitely for new work on workerChan, will not exit until workerChan is closed, is asleep whenever it waits for new work
• the dispatcher goroutine: Loops forever, selecting over two channels. Will never exit, is asleep while waiting in the select
• the main goroutine: Loops forever on a time.Sleep, will never exit and be asleep most of the time

Typically, in a situation like this, you'd introduce a chan struct{} (call it closing or something like that) and inclue it in your selects. If you want to close the program, just close(closing). The select will choose the <-closing option, you return the goroutines. You should also add a sync.WaitGroup to be notified when all your goroutines have exited.