read() after select() blocking when reading from pipe from spawned process

Question

There are 3 pipe calls to create stdin, stdout, stderr for a a new process. fork() is called, the exec() is called. This wrapped up in a popen2 function which works.

When using this popen2 function, reading from stdout from the new process blocks on read(), even after select() returns that read is ready to read. I expect it to be able to read(). My only guess is that read() for this fd is trying to fill the input buf.

One exception: if stdin is closed, stdout is closed by the child process and the read completes even when buf cannot be filled.

What I need is for read() to return what is ready to be read? is the default mode buffered and I don't know it?

pid_t popen2(const char *const argv[], int *in, int *out, int *err)
    {
    int res;
    pid_t pid = 0;
    int inpipefd[2];
    int outpipefd[2];
    int errpipefd[2];
    if(0!=pipe(inpipefd)) {
        perror("allocating pipe for child stdin");
        return -1;
    }
    if(0!=pipe(outpipefd)) {
        close(inpipefd[0]);
        close(inpipefd[1]);
        perror("allocating pipe for child stdout");
        return -1;
    }
    if(0!=pipe(errpipefd)) {
        close(inpipefd[0]);
        close(inpipefd[1]);
        close(outpipefd[0]);
        close(outpipefd[1]);
        perror("allocating pipe for child stderr");
        return -1;
    }
    pid = fork();
    if (0==pid) {
        if (-1==dup2(inpipefd[0], STDIN_FILENO)) {exit(errno);}
        if (-1==dup2(outpipefd[1], STDOUT_FILENO)) {exit(errno);}
        if (-1==dup2(errpipefd[1], STDERR_FILENO)) {exit(errno);}
        close(inpipefd[0]);
        close(inpipefd[1]);
        close(outpipefd[0]);
        close(outpipefd[1]);
        close(errpipefd[0]);
        close(errpipefd[1]);
        execvp(argv[0], (char* const*)argv);
        perror("exec failed");
        exit(1);
    }
    close(inpipefd[0]);
    close(outpipefd[1]);
    close(errpipefd[1]);
    *in = inpipefd[1];
    *out = outpipefd[0];
    *err = errpipefd[0];
    return pid;
    }

...
if(0>=(pid = popen2(argv, &in, &out, &err))) {
        return make_unique<std::string>();
    }
    res = writeall(in, cont.c_str(), cont.length());
    if(res==-1) {
        goto err;
    }
    close(in);
...

unique_ptr<std::string> DecryptProcess::Read() {
    auto result = make_unique<std::string>();
    const unsigned int BUFLEN = 1024*16;
    std::vector<char> buf(BUFLEN);
    fd_set rfds;
    struct timeval tv;
    int n;
    int fcnt;
    FD_ZERO(&rfds);
    FD_SET(out_fd, &rfds);
    FD_SET(err_fd, &rfds);
    tv.tv_sec = 0;
    tv.tv_usec = 100000;
    fcnt = select(std::max(out_fd, err_fd)+1, &rfds, NULL, NULL, &tv);
    if (fcnt == -1) {
        return result;
        } else if (!fcnt) {
        return result;
        }
    if (FD_ISSET(err_fd, &rfds)) {
        n = read(err_fd, &buf[0], buf.size());
        }
    if (FD_ISSET(out_fd, &rfds)) {
        do
            {
            n = read(out_fd, &buf[0], buf.size());
            if (n == -1) {
                return result;
                }
            if (n>0)
                result->append(buf.cbegin(), buf.cbegin()+n);
            } while ( n > 0 );
        }
    return result;
    }

Answer 1

[deleted the debugging statements which only clutter the view]

    if (FD_ISSET(out_fd, &rfds)) {
        do
            {
            n = read(out_fd, &buf[0], buf.size());
            if (n == -1) {
                return result;
                }
            if (n>0)
                result->append(buf.cbegin(), buf.cbegin()+n);
            } while ( n > 0 );
        }

Here you're not doing a single read(), but doing the read() in a loop until it returns an error or hits EOF. So code will loop until it will consume all data already written to the pipe, and then block until more data is written to it. select() returning a file as ready for reading only tells that there's some data available to be read from it, not that no read() will block until EOF.

And btw, select() makes no guarantees that even a single read() on a fd marked as ready for reading will not actually block.^[1] The only way to be sure of that is to set the fd to non-blocking mode (eg. with fcntl(O_NONBLOCK)) and check for errno == EAGAIN in the error case.

Your code has many other problems (eg. double flushing of stdio buffers, failure to check for EINTR, etc).

---

^[1] See the BUGS section of the Linux man page.