CODEWITHSUNDEEP
goconcurrency
nagrom97
nagrom97

Reputation: 514

Concurrency example confusion

I’m studying concurrency in more depth with Go. A book I am reading gives the following example, which basically compresses files from command line argument(s).

 package main

 import (
  "compress/gzip"
  "io"
  "os"
 )

 func main() {

     for _, file := range os.Args[1:] {
        compress(file)
     }

 }


 func compress(filename string) error {

     in, err := os.Open(filename)

     if err != nil {
      return err
     }

     defer in.Close()

     out, err := os.Create(filename + ".gz")

     if err != nil {
       return err
     }

     defer out.Close()

     gzout := gzip.NewWriter(out)

     _, err = io.Copy(gzout, in)

     gzout.Close()

     return err
}

The book then explains that if you wanted to process a few hundred files, keeping it like this would definitely be slower than if you were to utilise goroutines, so the following modification is made to the main() function in order to use them: 

var wg sync.WaitGroup 
var i int = -1
var file string
for i, file = range os.Args[1:] {
    wg.Add(1)
    go func(filename string) {
       compress(filename)
       wg.Done()
     }(file)
 }
 wg.Wait()
 fmt.Printf("Compressed %d files\n", i+1)

It is then noted that the “trickery” with regard to the inline function definition and its parameter (file name), is necessary “because we are executing goroutines in a for loop”.

I guess what I don’t understand is why the above inline function is required to get this working, couldn’t the following be used instead, or am I missing a trick?

for i, file = range os.Args[1:] {
    wg.Add(1)

    go compress(file)

    wg.Done()

 }
 wg.Wait()
 fmt.Printf("Compressed %d files\n", i+1)

Upvotes: 0

Views: 89

Answers (1)

eugenioy
eugenioy

Reputation: 12393

If you just execute:

compress(file)

That gets executed in the same goroutine, so until that call finishes, you won't be doing anything else.

So if you want to process in parallel, then you need to launch a new goroutine by using the go keyword.

If you do this:

go compress(file)

Then it will launch a new goroutine for compressing each file.

But, as soon as the main function launches all goroutines, it will end execution (it will not wait for all goroutines to finish).

That's why they included the wg calls.

Since the compress function does not receive a WaitGroup to call the Done over it, they implemented that inline function in order to call wg.Done() right after that goroutine finishes the compression over that file.

That's the reason for the:

go func(filename string) {
    compress(filename)
    wg.Done()
}(file)

Upvotes: 4

Related Questions