Keeping double quotes when passing string to popen in C

Question

I'm having a dilemma where I'm trying to pass a string through popen in C, but have it maintain the double quotes in the string. The string reads as follows:

ssh %s@%s grep -c \"%s\" %s%s

I need to run this command so it greps from a log on a remote system and returns the count for it. I'm passing different arguments to through the string, so it generates the username and password as well as the search string and target. However, the search string contains multiple words separated by whitespace characters, so I need the double quotes intact so it parses out the search string correctly. However, so far popen has been stripping the double quotes from the string so the search doesn't complete. I'm not sure of another way to do the remote ssh command, or of keeping the double quotes in the statement. Any ideas?

Thanks!

*EDIT: Here is the full sprintf statement I'm using to generate the string as well as the popen command.

sprintf(command, "ssh %s@%s grep -c \"%s\" %s%s", user, server, search, path, file);
fd = popen(command, "r");

Answer 1

popen forks a child which execs a shell with 2 arguments: -c and the command string you passed to popen. So any characters you pass that are meaningful to the shell need to be escaped so the shell does the right thing with them. In your case, you need the shell to KEEP the " characters so that the remote shell will get them, which is easiest to do by wrapping ' quotes around them:

sprintf(command, "ssh %s@%s grep -c '\"%s\"' %s%s", ...

However, this will only work if your search string does not contain any ' or " characters -- if it does, you need to do some more complex escaping. You could instead use backslash escapes:

sprintf(command, "ssh %s@%s grep -c \\\"%s\\\" %s%s", ...

but THIS will fail if your search string has quotes or multiple consecutive spaces or other whitespace like tabs int it. To deal with all cases you need first insert backslashes before all relevant charaters in the search string, plus ' are a pain to deal with:

// allocate 4*strlen(search)+1 space as escaped_search
for (p1 = search, p2 = escaped_search; *p1;) {
    if (*p1 == '\'')
        *p2++ = '\'';
    if (strchr(" \t\r\n\"\\'{}()<>;&|`$", *p1))
        *p2++ = '\\';
    if (*p1 == '\'') 
        *p2++ = '\'';
    *p2++ = *p1++;
}
*p2 = '\0';
sprintf(command, "ssh %s@%s grep -c '%s' %s%s", user, server, escaped_search, ...

Answer 2

As others have explained here, quoting sometimes gets cumbersome.

To avoid excessive quoting just pipe the commands into the standard input of ssh. Like in the following bash code:

ssh remote_host <<EOF
grep -c "search string" path
EOF

Answer 3

After some experimentation, this seems likely to be what you need:

snprintf(command, sizeof(command), "ssh %s@%s grep -c \\\"%s\\\" %s%s",
         username, hostname, search_string, directory, file);

You need multiple backslashes because multiple interpreters of backslashes are involved.

1. The C compiler: it treats the first two backslashes in each sequence of three as one backslash. The third backslash escapes the double quote, embedding a double quote in the command string. The command string then contains \" twice.
2. There is another process processing the backslashes, and it is a bit tricky to decide where it is, but they are needed to get the correct result, as shown by experimentation.

Working Code — Remote Execution, Right Result

Here is some demo code working. The local program is called pop. Everything in there is hard-wired because I'm lazy (but I munged the remote hostname compared with the one I actually tested with). The program /u/jleffler/linux/x86_64/bin/al lists its arguments as received, one per line. I find it a very useful tool for situations like this. Note that the arguments to al are carefully crafted with double spaces to show when the arguments are treated as one vs many.

$ ./pop
Command: <<ssh jleffler@remote.example.com /u/jleffler/linux/x86_64/bin/al \"x  y  z\" \"pp qq rr\">>
jleffler@remote.example.com's password: 
Response: <<x y z>>
Response: <<pp qq rr>>
$

Code

#include <stdio.h>
#include <string.h>

int main(void)
{
    char command[512];
    char const *arg1 = "x  y  z";
    char const *arg2 = "pp qq rr";
    char const *cmd  = "/u/jleffler/linux/x86_64/bin/al";
    char const *hostname = "remote.example.com";
    char const *username = "jleffler";

    snprintf(command, sizeof(command), "ssh %s@%s %s \\\"%s\\\" \\\"%s\\\"",
             username, hostname, cmd, arg1, arg2);

    printf("Command: <<%s>>\n", command);
    FILE *fp = popen(command, "r");
    char line[512];
    while (fgets(line, sizeof(line), fp) != 0)
    {
        line[strlen(line)-1] = '\0';
        printf("Response: <<%s>>\n", line);
    }
    pclose(fp);
    return(0);
}

Variant 1 — Remote Execution, Wrong Result

$ ./pop1
Command: <<ssh jleffler@remote.example.com /u/jleffler/linux/x86_64/bin/al "x  y  z" "pp qq rr">>
jleffler@remote.example.com's password: 
Response: <<x>>
Response: <<y>>
Response: <<z>>
Response: <<pp>>
Response: <<qq>>
Response: <<rr>>
$

Code

#include <stdio.h>
#include <string.h>

int main(void)
{
    char command[512];
    char const *arg1 = "x  y  z";
    char const *arg2 = "pp qq rr";
    char const *cmd  = "/u/jleffler/linux/x86_64/bin/al";
    char const *hostname = "remote.example.com";
    char const *username = "jleffler";

    snprintf(command, sizeof(command), "ssh %s@%s %s \"%s\" \"%s\"",
             username, hostname, cmd, arg1, arg2);

    printf("Command: <<%s>>\n", command);
    FILE *fp = popen(command, "r");
    char line[512];
    while (fgets(line, sizeof(line), fp) != 0)
    {
        line[strlen(line)-1] = '\0';
        printf("Response: <<%s>>\n", line);
    }
    pclose(fp);
    return(0);
}

Variant 2 — Local Execution, (Different) Wrong Result

$ ./pop2
Command: <<al jleffler@remote.example.com /u/jleffler/linux/x86_64/bin/al \"x  y  z\" \"pp qq rr\">>
Response: <<jleffler@remote.example.com>>
Response: <</u/jleffler/linux/x86_64/bin/al>>
Response: <<"x>>
Response: <<y>>
Response: <<z">>
Response: <<"pp>>
Response: <<qq>>
Response: <<rr">>
$

The local shell doesn't need the backslash before the double quote; in fact, adding it gets it wrong.

Code

#include <stdio.h>
#include <string.h>

int main(void)
{
    char command[512];
    char const *arg1 = "x  y  z";
    char const *arg2 = "pp qq rr";
    char const *cmd  = "/u/jleffler/linux/x86_64/bin/al";
    char const *hostname = "remote.example.com";
    char const *username = "jleffler";

    snprintf(command, sizeof(command), "al %s@%s %s \\\"%s\\\" \\\"%s\\\"",
             username, hostname, cmd, arg1, arg2);

    printf("Command: <<%s>>\n", command);
    FILE *fp = popen(command, "r");
    char line[512];
    while (fgets(line, sizeof(line), fp) != 0)
    {
        line[strlen(line)-1] = '\0';
        printf("Response: <<%s>>\n", line);
    }
    pclose(fp);
    return(0);
}

Variant 3 — Local Execution, Right Result

$ ./pop3  
Command: <<al jleffler@remote.example.com /u/jleffler/linux/x86_64/bin/al "x  y  z" "pp qq rr">>
Response: <<jleffler@remote.example.com>>
Response: <</u/jleffler/linux/x86_64/bin/al>>
Response: <<x  y  z>>
Response: <<pp qq rr>>
$

Code

#include <stdio.h>
#include <string.h>

int main(void)
{
    char command[512];
    char const *arg1 = "x  y  z";
    char const *arg2 = "pp qq rr";
    char const *cmd  = "/u/jleffler/linux/x86_64/bin/al";
    char const *hostname = "remote.example.com";
    char const *username = "jleffler";

    snprintf(command, sizeof(command), "al %s@%s %s \"%s\" \"%s\"",
             username, hostname, cmd, arg1, arg2);

    printf("Command: <<%s>>\n", command);
    FILE *fp = popen(command, "r");
    char line[512];
    while (fgets(line, sizeof(line), fp) != 0)
    {
        line[strlen(line)-1] = '\0';
        printf("Response: <<%s>>\n", line);
    }
    pclose(fp);
    return(0);
}

Answer 4

The problem is not with sprintf and not with popen. The problem is the shell that invokes ssh. It is the shell that strips your quotes.

You can simply open a terminal and try it manually. You will see that

ssh user@server grep -c "search string" path

does not work as intended if the search string contains spaces. Your shell consumes the quotes, so that in the end grep receives its command line without quotes and interprets it incorrectly.

If you want the quotes to persist, you have to escape them for the shell. The command you want to execute is

ssh user@server grep -c \"search string\" path

To form such a string using sprintf you also have to escape the " and \ characters (for C compiler) meaning that \" turns into \\\". The final format line is as follows

sprintf(command, "ssh %s@%s grep -c \\\"%s\\\" %s%s", user, server, search, path, file);

Of course, this can still suffer from other issues mentioned in Chris Dodd's answer.