writing to external program through pipe (C)

Question

I wanted to use external program to process data in memory. Like external compressor, encoder, anything to process my data and get the result. I read a lot about pipes and it still din't work. So I ended up with simple program that tries to write to external program through pipe like this, letting it to print to stdout:

                                                     stdout
             (w) pipeA (r)          $prog            +---+
             +-----------+       /~~~~~~~~~~~\       |{1}|
             |[1]     [0]| ----> |{0}     {1}| ----> |   |
        +~~> +-----------+       \~~~~~~~~~~~/       |   |
        |                                            +---+
        |
       +-+
 write() |
       +-+

And I still got nowhere. My code goes like this:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>


int main(void)
{
    int pipA[2];
    int pid;
    char buf_IN[32] = "Hello pipe!\n";
    ssize_t n_written;

    if ((pipe(pipA) == -1))    {
        perror("pipe failed");
        exit(1);
    }
    if ((pid = fork()) < 0)    {
        perror("fork failed");
        exit(2);
    }
    /*****************************/

    if (pid == 0)
        { /* in child */
        dup2(0, pipA[0]);   // pipA[read(0)-end]->$prog[write{0}-end]
        close(pipA[1]);   // $prog won't write to this pipe(A)
        // external ``$prog''ram
        execlp("wc", "wc", (char *) 0);   // out should be: '      1       2      12'
        //execlp("base64", "base64", (char *) 0);   // out should be: 'SGVsbG8gcGlwZSEK'

        ;///if we're here something went wrong
        perror("execlp() @child failed");
        exit(3);
        }

    else
        { /* in parent */
        //dup2(pipA[1], 0);  // STDIN -> pipA // that supposed to connect STDIN->pipA; just in case I needed it
        close(pipA[0]);  // we won't read it, let $prog write to stdout
        //perror("execlp() @parent failed");
        //exit(4);
        n_written = write(pipA[1], buf_IN, strlen(buf_IN));
        close(pipA[1]);  // I guess this will close the pipe and send child EOF

        // base64: read error: Input/output error
        // wc: 'standard input': Input/output error
        //      0       0       0
        }

return 0;
}

Comments show what I'm doing. I have to admit I don't get these dup()s in pipes and that's what I think is causing a problem here but don't know. Can you help with this, seemingly simple problem? Any help appreciated.

Answer 1

Diagnosis

You have the arguments to dup2() back-to-front. You need:

dup2(pipA[0], 0);

Closing file descriptors

You are not closing enough file descriptors in the child:

---

Rule of thumb: If you dup2() one end of a pipe to standard input or standard output, close both of the original file descriptors returned by pipe() as soon as possible. In particular, you should close them before using any of the exec*() family of functions.

The rule also applies if you duplicate the descriptors with either dup() or fcntl() with F_DUPFD

---

Prescription

You have some unused defines and unused variables in your code, too. Shorn of all your comments (but with a few of mine to explain what's happening) and with appropriate fixes in place, I end up with:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(void)
{
    int pipA[2];
    int pid;
    char buf_IN[32] = "Hello pipe!\n";
    ssize_t n_written;

    if ((pipe(pipA) == -1))
    {
        perror("pipe failed");
        exit(1);
    }
    if ((pid = fork()) < 0)
    {
        perror("fork failed");
        exit(2);
    }

    if (pid == 0)
    {
        /* Child: Connect pipA[0] (read) to standard input 0 */
        dup2(pipA[0], 0);
        close(pipA[1]);  /* Close write end of pipe */
        close(pipA[0]);  /* Close read  end of pipe */
        execlp("wc", "wc", (char *)0);
        perror("execlp() @child failed");
        exit(3);
    }
    else
    {
        close(pipA[0]);  /* Close read end of pipe */
        n_written = write(pipA[1], buf_IN, strlen(buf_IN));
        if (n_written != (ssize_t)strlen(buf_IN))
        {
            perror("short write");
            exit(4);
        }
        close(pipA[1]);  /* Close write end of pipe — EOF for child */
    }

    /* Optionally wait for child to die before exiting */
    // #include <sys/wait.h>  // With other #include lines
    // int corpse;
    // int status;
    // while ((corpse = wait(&status)) > 0)
    //     printf("Child %d exited with status 0x%.4X\n", corpse, status);

    return 0;
}

When run, that produces:

       1       2      12

That looks about right.

Without the wait() loop, it is possible that you'll see the output from wc after the prompt from the shell (so it might look as if the program is waiting for input from you, but in fact, it will be the shell waiting for input); with the waiting loop, you'll get proper separation of output from the shell prompt. You don't have to print anything in the body of the loop, but it is reassuring to do so.

Answer 2

To not make duplicates, nor to play expert in unfamiliar field I post it as answer.

I finished the task

                 (w) pipeA (r)            child            (w) pipeB (r)
                 +-----------+       /~~~~~~~~~~~~~\       +-----------+
            +~~~>|[1]     [0]| ----> |{0} $prog {1}| ----> |[1]     [0]| ~~~+
            |    +-----------+       \~~~~~~~~~~~~~/       +-----------+    |
            |                                                               \/
           +-+                                                             +--+
     write() |                                                             |  read()
           +-+                                                             +--+

                dup(pA[0],0)                                dup(pB[1],1)
                close(pA[1])                                close(pA[0])
                close(pA[0])                                close(pA[1])

with second pipe that can be read. All by analogy. If there is something major that is wrong with it or something I should be aware of, say it please. (Sorry for python style indentations, hope you don't mind)

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

ssize_t read_whole_file(int fildes, const void *buf, size_t nbyte);

int main(void)
    {
    int pipA[2], pipB[2];
    int pid;
    char buf_IN[32] = "Hello pipe!\n";
    char buf_OUT[1024];
    char *bptr;
    ssize_t n_written, n_read = 0, a_read = 0, to_read = sizeof(buf_OUT);

    if ((pipe(pipA) == -1) || (pipe(pipB) == -1))
        {
        perror("pipe failed");
        exit(1);
        }
    if ((pid = fork()) < 0)
        {
        perror("fork failed");
        exit(2);
        }

    if (pid == 0)
        {
        /* in child */
        dup2(pipA[0], 0);  // connect pipe A (read end/exit) to stdin (write end/input)
        close(pipA[1]);  // close unused pipe A end
        close(pipA[0]);  // close  - " - //
        ;
        dup2(pipB[1], 1);  // connect stdout (read end/output) to pipe B (write end/entry)
        close(pipB[0]);  // close unused pipe B ends
        close(pipB[1]);  // close  - " - //
        execlp("lzip", "lzip", "-c", (char *)0);
        ;
        perror("execlp() @child failed");
        exit(3);
        }
    else
        {
        /* in parent */
        close(pipA[0]);  // close pipe A read end - will only write to this one
        n_written = write(pipA[1], buf_IN, strlen(buf_IN));
        if (n_written < 0)
            perror("error: read_whole_file(pipA[1], ...) failed miserably\n");
        close(pipA[1]);  // close write end which subsequently signals EOF to child
        ;
        close(pipB[1]);  // close pipe B write end - will only read form this one
        a_read = read_whole_file(pipB[0], buf_OUT, sizeof(buf_OUT));
        if (a_read < 0)
            perror("error: read_whole_file(pipB[0], ...) failed miserably\n");
        close(pipB[0]);  // close read end after reading
        ;
        write(STDOUT_FILENO, buf_OUT, a_read);  // dump it to parent's stdout - equivalent of processing data received from external program/plugin
        }
    return 0;
    }

ssize_t read_whole_file(int fildes, const void *buf, size_t nbyte)  // read whole file
    {
    ssize_t n_read, a_read = 0, to_read = nbyte;
    char *bptr = (char*)buf;
    size_t BUF_SIZE = 4096;
    do
        {
        if(to_read < BUF_SIZE)
            BUF_SIZE = to_read;
        n_read = read(fildes, bptr, BUF_SIZE);
        if (n_read < 0)  // recover from temporarily failed read (got it from git wrapper)
            {
            if ((errno == EINTR) || (errno == EAGAIN) || (errno == EWOULDBLOCK))
                continue;
            }
        bptr += n_read;
        to_read -= n_read;
        a_read += n_read;
        } while ((n_read>0) && (to_read>0));
    ;
    if (n_read < 0)
        a_read = n_read;
    return a_read;
    }

To note not so obvious - I still don't get close(pipA[0]); and close(pipB[1]); in child. Why are they not used anymore? Also dup2(pipB[1], 1);, I thought it would be other way around but it didn't work so by trial end error I come with this.