CODEWITHSUNDEEP
concurrencygogoroutine
Danny Dyla
Danny Dyla

Reputation: 691

goroutines causing major slowdowns and headaches

I'm having something of a problem with goroutines. Why is it that this code executes in ~125ms (note sequential execution):

package main

import (
  "os/exec"
  "time"
  "fmt"
)

func main() {
  cmd := exec.Command("lessc", "--yui-compress", "test.less")
  n := 2000
  start := time.Now()
  for i := 0; i < n; i++ {
    cmd.Run()
  }
  finish := time.Now()

  fmt.Printf("Program took %v to run\n", finish.Sub(start))
}

When this code takes about 20 seconds (concurrent execution using goroutines):

package main

import (
  "os/exec"
  "time"
  "fmt"
)

func main() {
  cmd := exec.Command("lessc", "--yui-compress", "test.less")
  ch := make(chan bool)
  n := 2000
  start := time.Now()
  for i := 0; i < n; i++ {
    go lessc(ch, cmd)
  }
  fmt.Println(n, " goroutines started.")
  for i := 0; i < n; i++ {
    _ = <-ch
  }
  finish := time.Now()

  fmt.Printf("Program took %v to run\n", finish.Sub(start))
}

func lessc(ch chan bool, c *exec.Cmd) {

  c.Run()
  ch <- true
}

Using go 1.0.3 on i7 720QM (4C/8T) 8GB RAM linux/x86-64 Also built and tested using 1.0.2 and got the same problem on the same machine.

Edit: Solved by @jnml below. If anyone cares about the new fixed concurrent code here it is:

package main

import (
  "os/exec"
  "time"
  "fmt"
)

func main() {
  ch := make(chan bool)
  n := 2000
  start := time.Now()
  for i := 0; i < n; i++ {
    go lessc(ch)
  }
  fmt.Println(n, " goroutines started.")
  for i := 0; i < n; i++ {
    _ = <-ch
  }
  finish := time.Now()

  fmt.Printf("Program took %v to run\n", finish.Sub(start))
}

func lessc(ch chan bool) {

  cmd := exec.Command("lessc", "--yui-compress", "test.less")
  cmd.Run()
  ch <- true
}

Upvotes: 1

Views: 211

Answers (1)

zzzz
zzzz

Reputation: 91409

IMO your program is not correct. It contains a race condition and thus can do literally anything. Any timing of it don't make sense.

You're creating one exec.Cmd and then concurrently (== data race) perform its Run method from several goroutines. exec.Cmd never mentions it could be reused for more than once Run - even if serially.

exec.Cmd has some state initialized by exec.Command and a different state after executing Run. IOW, after executing the Run method, the state is not anymore initialized and probably not fit for another Run

Upvotes: 11

Related Questions