Go Channel and go routine with pointer variables all goroutines are asleep - deadlock

Question

I spend my evening looking at how to fix this error but I haven't succeeded. When I run the program I have the following error : "all goroutines are asleep - deadlock!". I understand this is because the main program is exited before the routine has the possibility to do its tasks and I thought using sync.WaitGroup would help but not really :/

I want to set a number of routine and use channels to send urls in order to check http status code. I want to limit the number of concurrency call to a website. I've followed examples doing the same thing with string instead of struct and it worked.

Any help would be well appreciated :)

package main

import (
    "fmt"
    "sync"
    "time"
)

const (
    numPollers = 2                // number of Poller goroutines to launch
)

var urls = []string{
    "http://www.google.com/",
    "http://golang.org/",
    "http://blog.golang.org/",
    "http://golangtutorials.blogspot.fr",
    "https://gobyexample.com/",
}

// Resource represents an HTTP URL to be polled by this program.
type Resource struct {
    url      string
}

func Poller(in <-chan *Resource, wg *sync.WaitGroup) {
    //defer wg.Done()
    for r := range in {
        fmt.Printf("Finished: %v - %v\n", r.url, time.Now())
    }
    wg.Done()
}

func main() {
    var wg sync.WaitGroup
    pending := make(chan *Resource)

    wg.Add(len(urls))

    go Poller(pending, &wg)

    go func() {
        for _, url := range urls {
            wg.Add(1)
            fmt.Println("SENT > Pending url " + url)
            pending <- &Resource{url: url}
        }
    }()

    wg.Wait()

    fmt.Printf("Finished all goroutines: %v\n", time.Now())
}

https://play.golang.org/p/B-HSiDo2Qg

Answer 1

First, you have too many calls to wg.Add(). You call that once for each goroutine you're running. See http://golang.org/pkg/sync/#WaitGroup. Second, you didn't close the channel after you were done writing to it. Here's a modified version of your code:

package main

import (
    "fmt"
    "sync"
    "time"
)

const (
    numPollers = 2                // number of Poller goroutines to launch
)

var urls = []string{
    "http://www.google.com/",
    "http://golang.org/",
    "http://blog.golang.org/",
    "http://golangtutorials.blogspot.fr",
    "https://gobyexample.com/",
}

// Resource represents an HTTP URL to be polled by this program.
type Resource struct {
    url      string
}

func Poller(in <-chan *Resource, wg *sync.WaitGroup) {
    defer wg.Done()
    for r := range in {
        fmt.Printf("Finished: %v - %v\n", r.url, time.Now())
    }
}

func main() {
    var wg sync.WaitGroup
    pending := make(chan *Resource)

    wg.Add(2)

    go Poller(pending, &wg)

    go func() {
        defer close(pending)
        defer wg.Done()
        for _, url := range urls {
            fmt.Println("SENT > Pending url " + url)
            pending <- &Resource{url: url}
        }
    }()

    wg.Wait()

    fmt.Printf("Finished all goroutines: %v\n", time.Now())
}

and https://play.golang.org/p/ucUlZEZMZM

Answer 2

You forget to close channel, and your wait group is too long. This works for me: https://play.golang.org/p/yasIzaCbmQ