CODEWITHSUNDEEP
go
Bryce Thomas
Bryce Thomas

Reputation: 10799

writing large amounts of data to exec.Command().StdinPipe() errors

Consider the following Go code fragment:

cmd := exec.Command(program, arg0)
stdin, err := cmd.StdinPipe()
// produces error when b is too large
n, err := stdin.Write(b.Bytes())

Whenever b is too large, Write() returns an error. Having experimented with different size bs, it would seem this occurs whenever the length of b is longer than the Linux pipe buffer size. Is there a way around this? Essentially I need to feed large log files via stdin to an external script.

Upvotes: 2

Views: 572

Answers (1)

fuz
fuz

Reputation: 92984

I wrote this program to test your code:

package main

import "os/exec"
import "fmt"


func main() {
        cmd := exec.Command("/bin/cat")

        in, _ := cmd.StdinPipe()

        cmd.Start()

        for i := 1024*1024; ; i += 1024*1024 {
                b := make([]byte,i)
                n, err := in.Write(b)
                fmt.Printf("%d: %v\n", n, err)
                if err != nil {
                        cmd.Process.Kill()
                        return
                }
        }
}

The only way this program gives an error is if the called process closes stdin. Does the program you call close stdin? This might be a bug in the Go runtime.

Upvotes: 1

Related Questions