CODEWITHSUNDEEP
cstringtokenize
user267895
user267895

Reputation: 31

String tokenizing problems

I have this string: "Alaska:(3,4)" and I want "Alaska", "3", "4" sub-strings. But I have a problem.

cityName = strtok(str , ":");
printf("name : %s\n",cityName);
temp = strtok(NULL , "(");
printf("%s\n",temp);
temp = strtok(NULL , ",");
printf("%s\n",temp);
temp = strtok(NULL, ")");
printf("%s\n",temp);

For this code I get this output:

name : Alaska
3,4)
(null)
(null)

What's wrong?

Upvotes: 2

Views: 110

Answers (3)

Jeffery Thomas
Jeffery Thomas

Reputation: 42588

strtok(3) doesn't like empty separators. Do you have access to strsep(3).

char str[] = "Alaska:(3,4)";
char *p = str;
char *temp;
char *cityName = strsep(&p , ":");
printf("name : %s\n",cityName);
temp = strsep(&p , "(");
printf("%s\n",temp);
temp = strsep(&p , ",");
printf("%s\n",temp);
temp = strsep(&p, ")");
printf("%s\n",temp);

Upvotes: 0

Deduplicator
Deduplicator

Reputation: 45654

Please take a look at the documentation for strtok:

A sequence of calls to the strtok function breaks the string pointed to by s1 into a sequence of tokens, each of which is delimited by a character from the string pointed to by s2. The first call in the sequence has a non-null first argument; subsequent calls in the sequence have a null first argument. The separator string pointed to by s2 may be different from call to call.
The first call in the sequence searches the string pointed to by s1 for the first character that is not contained in the current separator string pointed to by s2. If no such character is found, then there are no tokens in the string pointed to by s1 and the strtok function returns a null pointer. If such a character is found, it is the start of the first token.
The strtok function then searches from there for a character that is contained in the current separator string. If no such character is found, the current token extends to the end of the string pointed to by s1, and subsequent searches for a token will return a null pointer. If such a character is found, it is overwritten by a null character, which terminates the current token. The strtok function saves a pointer to the following character, from which the next search for a token will start.

In short, strtok searches for the first character not in delimiters for the start of the sequence, and then for the first character in delimiters for the end of the sequence.
That means your last two calls to strtok return NULL, so the printf-calls are Undefined Behavior, anything may happen.

Better use sscanf or roll your own parser (likely does not need any longer).

If you want to stay with strtok, correct your delimiters and only try to get three tokens:

cityName = strtok(str , ":");
printf("name : %s\n",cityName);
temp = strtok(NULL , "(,)");
printf("%s\n",temp);
temp = strtok(NULL , "(,)");
printf("%s\n",temp);

Still, consider at least moving to strtok_s to avoid data-races and make your code reentrant.

Upvotes: 0

Sergey L.
Sergey L.

Reputation: 22542

The reason why your implementation does not work is because strtok has recognised the string as fully parsed after the second call (because nothing was found before the first token). You need to restart the parsing after the second call.

void bar(char * str) {
    char * cityName, *temp;
    cityName = strtok(str , ":");
    printf("name : %s\n",cityName);
    temp = strtok(NULL , "(");
    printf("%s\n",temp);
    temp = strtok(temp , ","); // restart parsing here
    printf("%s\n",temp);
    temp = strtok(NULL, ")");
    printf("%s\n",temp);
}

Please note that strtok is destructive on the input string and is not thread-safe.

Have you considered to use sscanf family? It's a bit easier to use if you have a fixed format.

void foo(char * str) {
    char city[32], num[2][32];
    sscanf(str, "%[^:]:(%[^,],%[^)])", city, num[0], num[1]);
    printf("%s\n%s\n%s\n", city, num[0], num[1]);
}

Upvotes: 1

Related Questions