Passing the right argument to strtok

Question

I have this string that I'm trying to delimit with strtok():

"+ 2 3\\nln 9\\ln 10"

And then I'm passing \\n as a delimiter

    token = strtok(argv, "\\n");

    while (token)
    {
            args[index] = token;
            token = strtok(NULL, "\\n");
            index++;
    }

First element in my args table is + 2 3 ,which is great, however, the second one is l. Does anyone understand why ? If so, how do I get my ln 9 in my args table?

Answer 1

I completely agree with above answer and suggestions by Roberto and Gerhardh.

In case if you are fine with custom implementation of strtok for multiple delimeter, you can use below working solution.

char *strtokm(char *str, const char *delim)
{
    static char *tok;
    static char *next;
    char *m;

    if (delim == NULL) return NULL;

    tok = (str) ? str : next;
    if (tok == NULL) return NULL;

    m = strstr(tok, delim);

    if (m) {
        next = m + strlen(delim);
        *m = '\0';
    } else {
        next = NULL;
    }

    return tok;
}

Answer 2

From strtok() manpage:

The delim argument specifies a set of bytes that delimit the tokens in the parsed string. The caller may specify different strings in delim in successive calls that parse the same string.

So, in your code, "\\n" is not a full string delimiter. You are just saying to strtok that the delimiter is either '\' (because of the double backspace escaping) or 'n'.

The tokens of your string, "+ 2 3\\nln 9\\ln 10" will be:

1. "+ 2 3"
2. empty string between \ and \ (strtok doesn't present it)
3. empty between \ and n (strtok doesn't present it)
4. "l"
5. " 9"
6. empty string between \ and \ (strtok doesn't present it)
7. "l"
8. " 10"

In order to perform what you are trying to do, strtok is not the best choice. I would probably write my own parsering function

1. Finding "\\n" occurrences in original string using strstr()
2. Either copying the previous string to some output string or null terminating it in place