CODEWITHSUNDEEP
go
Louis Ng
Louis Ng

Reputation: 553

Running external python in Golang, Catching continuous exec.Command Stdout

So my go script will call an external python like this

cmd = exec.Command("python","game.py")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
go func(){
    err := cmd.Run()
    if err != nil{
    panic(err)
    }
}()

It runs my python script concurrently which is awesome. But now the problem is, my python script will run infinitely and it will print out some information from time to time. I want to "catch" these Stdout and print them out on my golang terminal. How do I do it concurrently (without waiting my python script to exit)?

Upvotes: 5

Views: 11456

Answers (3)

Damien D
Damien D

Reputation: 11

If you want to capture stdout and stderr concurrently, indeed you need to use cmd.StdoutPipe(), cmd.StderrPipe(), cmd.Start() and cmd.Wait().

But as mentioned in the documentation (https://pkg.go.dev/os/exec#Cmd.StdoutPipe):

It is thus incorrect to call Wait before all reads from the pipe have completed.

You'll need to wait for you goroutines to finish before calling cmd.Wait() (see the following thread: https://github.com/golang/go/issues/38268)

Upvotes: 1

hyman zhang
hyman zhang

Reputation: 121

You can use add the flag -u to turn off buffering,

cmd := exec.Command("python", "-u", "game.py")

https://docs.python.org/3/using/cmdline.html#cmdoption-u

https://bugs.python.org/issue526382

Upvotes: 1

minamijoyo
minamijoyo

Reputation: 3445

Use cmd.Start() and cmd.Wait() instead of cmd.Run().

https://golang.org/pkg/os/exec/#Cmd.Run

Run starts the specified command and waits for it to complete.

Start starts the specified command but does not wait for it to complete.

Wait waits for the command to exit. It must have been started by Start.

And if you want to capture stdout/stderr concurrently, use cmd.StdoutPipe() / cmd.StderrPipe() and read it by bufio.NewScanner()

package main

import (
    "bufio"
    "fmt"
    "io"
    "os/exec"
)

func main() {
    cmd := exec.Command("python", "game.py")
    stdout, err := cmd.StdoutPipe()
    if err != nil {
        panic(err)
    }
    stderr, err := cmd.StderrPipe()
    if err != nil {
        panic(err)
    }
    err = cmd.Start()
    if err != nil {
        panic(err)
    }

    go copyOutput(stdout)
    go copyOutput(stderr)
    cmd.Wait()
}

func copyOutput(r io.Reader) {
    scanner := bufio.NewScanner(r)
    for scanner.Scan() {
        fmt.Println(scanner.Text())
    }
}

The following is a sample python code for reproducing real-time output. The stdout may be buffered in Python. Explicit flush may be required.

import time
import sys

while True:
    print "Hello"
    sys.stdout.flush()
    time.sleep(1)

Upvotes: 8

Related Questions