CODEWITHSUNDEEP
goconcurrencygoroutine
Severin
Severin

Reputation: 1075

How to handle errors and terminate Goroutine using WaitGroup

I have been playing around with Goroutines, Channels and WaitGroup today and I am finally starting to understand the concept, after just been reading about it for a while.

My problem is that I am unsure how I handle errors when working like this, mainly because of the WaitGroup I use. When using the WaitGroup, I start by adding the amount of goroutines that will be executed, but what if an error happens during one of these? 

package main

import (
    "errors"
    "sync"
)

var waitGroup sync.WaitGroup

func main() {
    c := make(chan int, 10)

    waitGroup.Add(10)

    go doSomething(c)

    waitGroup.Wait()
}

func doSomething(c chan int) {
    for i := 0; i < 10; i++ {
        n, err := someFunctionThatCanError()

        if err != nil {
            // How do I end the routines and WaitGroups here?
        }

        c <- n
        waitGroup.Done()
    }

    close(c)    
}

func someFunctionThatCanError() (int, error) {
    return 1, errors.New("an error")
}

Playground: https://play.golang.org/p/ZLsBSqdMD49

I have tried my best to provide an example that shows what I am talking about. A loop will run 10 times in doSomething() and it will call waitGroup.Done() on every iteration, but what if an error happens during all this, like shown with someFunctionThatCanError()?

When I try to solve it now, by returning and/or cancelling the channel, I end up with deadlocks, so I am a little unsure where to go from here. I am also unsure of how to handel the WaitGroup that I assume is waiting for more things to happen.

Any help is really appreciated.

Upvotes: 7

Views: 14365

Answers (2)

Dima K
Dima K

Reputation: 1

errgroup.Group has several disadvantages:

  1. error did not return immediately, but only after the parallel goroutines were executed
  2. goroutine execution results must be collected manually

Pipers solves the problems described above:

import github.com/kozhurkin/pipers

func main() {
    ts := time.Now()
    urls := []string{
        "http://stackoverflow.com/questions/59095501",
        "http://golang.org",
        "http://google.com",
        "http://github.com",
    }

    ctx, cancel := context.WithTimeout(context.Background(), time.Second)
    defer cancel()

    pp := pipers.FromArgs(urls, func(i int, url string) (int, error) {
        resp, err := http.Get(url)
        if err != nil {
            return 0, err
        }
        return resp.StatusCode, nil
    })

    results, err := pp.Context(ctx).Concurrency(3).Resolve()

    fmt.Println(results, err, time.Since(ts))
    // [200 0 200 200] context deadline exceeded 1.00s
}

Pipers also provides some useful methods, such as .Concurrency(n) and .Context(ctx)

Upvotes: 0

Thundercat
Thundercat

Reputation: 121129

Use golang.org/x/sync/errgroup to wait on and handle errors from goroutines.

package main

import (
    "errors"
    "log"
    "sync"

    "golang.org/x/sync/errgroup"
)

func main() {
    c := make(chan int, 10)

    var g errgroup.Group

    g.Go(func() error {
        return doSomething(c)
    })

    // g.Wait waits for all goroutines to complete
    // and returns the first non-nil error returned
    // by one of the goroutines.
    if err := g.Wait(); err != nil {
        log.Fatal(err)
    }
}

func doSomething(c chan int) error {
    defer close(c)
    for i := 0; i < 10; i++ {
        n, err := someFunctionThatCanError()
        if err != nil {
            return err
        }
        c <- n
    }
    return nil
}

func someFunctionThatCanError() (int, error) {
    return 1, errors.New("an error")
}

Run it on the playground.

Upvotes: 18

Related Questions