CODEWITHSUNDEEP
goiogoroutine
roger
roger

Reputation: 9933

write string to file in goroutine

I am using go routine in code as follow:

c := make(chan string)
work := make(chan string, 1000)
clvl := runtime.NumCPU()

for i := 0; i < clvl; i++ {
    go func(i int) {
        f, err := os.Create(fmt.Sprintf("/tmp/sample_match_%d.csv", i))
        if nil != err {
            panic(err)
        }
        defer f.Close()
        w := bufio.NewWriter(f)
        for jdId := range work {
            for _, itemId := range itemIdList {
                w.WriteString("test")
            }
            w.Flush()
            c <- fmt.Sprintf("done %s", jdId)
        }
    }(i)
}


go func() {
    for _, jdId := range jdIdList {
        work <- jdId
    }

    close(work)
}()

for resp := range c {
    fmt.Println(resp)
}

This is ok, but can I all go routine just write to one files? just like this:

c := make(chan string)
work := make(chan string, 1000)
clvl := runtime.NumCPU()
f, err := os.Create("/tmp/sample_match_%d.csv")
if nil != err {
    panic(err)
}
defer f.Close()
w := bufio.NewWriter(f)

for i := 0; i < clvl; i++ {
    go func(i int) {

        for jdId := range work {
            for _, itemId := range itemIdList {
                w.WriteString("test")
            }
            w.Flush()
            c <- fmt.Sprintf("done %s", jdId)
        }
    }(i)
}

This can not work, error : panic: runtime error: slice bounds out of range

Upvotes: 0

Views: 1463

Answers (1)

Thundercat
Thundercat

Reputation: 121129

  • The bufio.Writer type does not support concurrent access. Protect it with a mutex.
  • Because the short strings are flushed on every write, there's no point in using a bufio.Writer. Write to the file directly (and protect it with a mutex).
  • There's no code to ensure that the goroutines complete before the file is closed or the program exits. Use a sync.WaitGroup.

Upvotes: 3

Related Questions