CODEWITHSUNDEEP
multithreadingcsvgo
Nick Mullen
Nick Mullen

Reputation: 117

How to solve "short write" error when writing to CSV

I've been encountering this issue that I can't seem to figure out what's happening. I have code that appends some information to a CSV. The information comes from multiple threads and the CSV is logging times.

At some point during program execution, the code will start spitting out "short write" instead of writing to the CSV.

Printing the data to be outputted, there seems to be no difference in "good" writes and "short" writes and once the "short write" error stops, it continues that until the program is restarted.

This code is called each time a CSV write is desired.

        output := []string{time.Message, strconv.FormatInt(time.Duration, 10)}
        fmt.Println(output)
        err := writer.Write(output)
        if err != nil {
            fmt.Println("An error encountered ::", err)
            fmt.Println("writer.Write error")
            logMessage(err.Error())
        }
        writer.Flush()

Looking at my output, this is the point where it starts the short write.

[message1 21]
[message2 207]
[message3 79]
An error encountered :: short write
writer.Write error
short write

And here's my output in the CSV

message1,21
message2,207
ssage2,207

So it seems like the short write is coming from that second line but why is it writing my message twice, and why is it "short writing" the message? There are hundreds of successful messages then the error just seems to occur suddenly.

Looking at the code, the place to call the CSV writer functionality only hits once, so I'm really stuck on why this is happening.

Upvotes: 3

Views: 3918

Answers (2)

Baron Lam
Baron Lam

Reputation: 47

I guess you are using writer with buffer(like bufio). This error would happen when a groutine is writing data to buffer and the other is trying to flush.

Just simply add a locker in the critical section.

mu.Lock()
// critical section
mu.Unlock()

Upvotes: 1

Thundercat
Thundercat

Reputation: 120999

The application has a data race on writer. Use a sync.Mutex to ensure that only one goroutine accesses the writer at a time.

mu.Lock() // <-- Lock to prevent concurrent write and flush.
output := []string{time.Message, strconv.FormatInt(time.Duration, 10)}
fmt.Println(output)
err := writer.Write(output)
if err != nil {
  fmt.Println("An error encountered ::", err)
  fmt.Println("writer.Write error")
  logMessage(err.Error())
}
writer.Flush()
mu.Unlock() // <-- Unlock.

Run the race detector on the application and fix any reported problems.

Upvotes: 3

Related Questions