CODEWITHSUNDEEP
gotcpforkexecnetstat
sahil garg
sahil garg

Reputation: 89

Getting "fork/exec /usr/bin/wc: resource temporarily unavailable"

I am trying to execute a netstat command to find out number of ports in use using golang,

I am using the following code for that,

func PortCheck() int {
    cmd := exec.Command("netstat", "-an")
    grep := exec.Command("grep", "8000")
    wc := exec.Command("wc", "-l")
    pipe, _ := cmd.StdoutPipe()
    defer pipe.Close()

    grep.Stdin = pipe
    pipe2, _ := grep.StdoutPipe()
    defer pipe2.Close()
    wc.Stdin = pipe2
    // Run ps first.
    cmd.Start()
    grep.Start()
    out, err := wc.Output()
    if err != nil {
        // if there was any error, print it here
        fmt.Println("could not run command: ", err)
    }
    // otherwise, print the output from running the command
    fmt.Println("Output: ", string(out))
    fmt.Println(cmd)

    ports, err := strconv.Atoi(strings.Split(string(out), "\n")[0])
    if err != nil {
        fmt.Println(err)
    }
    return ports
}

I am running this function concurrently, let's say for 10,000 iterations,

it starts giving error:-

fork/exec /usr/bin/wc: resource temporarily unavailable

Please help me out with the solution, Is this function will be more optimized, to find out the number of ports in use?

I am expecting to run the given function concurrently 1 million times.

Upvotes: 1

Views: 1333

Answers (1)

mpx
mpx

Reputation: 1364

Your implementation is not waiting for the processes to exit. After the ulimit for max user proceses is reached the program is unable to fork new processes.

You must call (*Cmd).Wait to wait for every process to exit before returning from the PortCheck implementation.

You should also connect your processes using os.Pipe. This is more or less how your shell implements pipes. Only use (*Cmd).StdoutPipe to read the result from the final command.

If you want to improve performance:

  • Implement grep 8000 | wc -l in Go. This can also be less buggy. Eg, I assume you don't want to match port 48000? Or perhaps you only want TCP, instead of UDP or unix sockets with "8000" in the inode?
  • ..or replace netstat with a specific Go implementation entirely. Eg, Linux netstat reads and interprets /proc/net/{tcp,tcp6,udp,...}. These files are easier to parse than the netstat output. It also avoids forking new processes and parsing irrelevant files. This can be massively faster. It is also less portable, but there can be subtle differences between OS netstat implementations so this function may need OS specific variations anyway.

Upvotes: 4

Related Questions