strange tokens results using strtok

Question

My external device sends me data every 5 seconds like these:

+DATA: 43 BYTES FROM 0000:0000 (043)
Nodo_8:(T=21.45,HR=45.65,DW=9.34,Vcc=3.46V)

I need some values from here to save them in a mysql datadase. So with the usage of strtok I want to get the values 043, 21.45, 45.65, 9.34, 3.46.

I wrote the following code which reads the buffer from the device:

int learn_port(int fd)
{
   int n;
   char buff[83];

 for (;;)
  {
    n=read(fd,buff,83);
    printf("%s", buff);
    char dev_a[25] = "", temp_a[25] = "", hr_a[25] = "", dw_a[25] = "", vcc_a[25] = "";
    char* ptr;

    ptr = strtok(buff, "+DATA:BYTESFROM()\nNodo_ ,=T:HR:DW:Vcc()");
    int i = 0;
   while (ptr != NULL)
   {
     ptr = strtok(NULL, "+DATA:BYTESFROM()\nNodo_ ,=T:HR:Dw:Vcc()");
     if (i == 2)
      strcat(dev_a, ptr); // copies device
     if (i == 5)
      strcat(temp_a, ptr); // copies T
     if (i == 6)
        strcat(hr_a, ptr); // copies HR
     if (i == 7)
        strcat(dw_a, ptr); // copies DW
     if (i == 10)
        strcat(vcc_a, ptr);
    i++;
    }
    sleep(1);
    printf("%s, %s, %s, %s, %s\n", dev_a, temp_a,hr_a,dw_a,vcc_a);
 }

But I have some strange results and I don't know where is there problem. The terminal returns me the first time:

+DATA: 43 BYTES FROM 0000:0000 (043)
Nodo_8:(T=21.45,HR=45.65,DW=9.34,Vcc=3.46V)
??,??043, 21.45, 45.65, 9.34, 3.46

after 5 secs

+DATA: 43 BYTES FROM 0000:0000 (043)
Nodo_8:(T=21.23,HR=42.65,DW=9.45,Vcc=3.46V)
?3.46043, 21.23, 42.65, 9.45, 3.46

after 5 secs

+DATA: 43 BYTES FROM 0000:0000 (051)
Nodo_8:(T=21.67,HR=42.45,DW=9.23,Vcc=3.46V)
?3.46051, 21.67, 42.45, 9.23, 3.46

etc. Does anyone know where is the problem and I have ?3.46 before the 051? Is there any problem with the strtok? My results want to be 043, 21.67, 42.45, 9.23, 3.46

Answer 1

Having helped before on this, I felt it's my duty to step in and try to sort out code of mine that's stopped doing what it's supposed to do :)

Okay, I suggest that you have two separate cases for your input, I've split them into ones beginning with '+' for '+DATA' and 'N' for 'Nodo_8' (this is very raw, make sure it is indeed the case otherwise have some further validation).

I've moved the call to strok from the start of the loop to the end as it may 'tokenise' the string twice at the first iteration before we've had the chance to extract anything. I've put it just before incrementing i. If you want to keep it at the start just subtract 1 from i where we do the checks.

Then it was just a matter of finding out at which iteration of the loop it would spit out the proper values.

#include <stdio.h>
#include <string.h>

int main()
{
    /* dev_a and dev_b could be the same source in your case, modify accordingly */
    char dev_a[] = "+DATA: 43 BYTES FROM 0000:0000 (043)";
    char dev_b[] = "Nodo_8:(T=21.23,HR=42.65,DW=9.45,Vcc=3.46V)";

    char out0[35] = "";
    char out1[35] = "";
    char out2[35] = "";
    char out3[35] = "";
    char out4[45] = "";

    char* ptr;
    int i;

    if (dev_a[0] == '+')
    {
        ptr = strtok(dev_a, "+DATA: BYTES FROM ()");

        i = 0;

        while (ptr != NULL)
        {            
            if (i == 3)
                strcat(out0, ptr); /* copies DATA (value in brackets) */

            ptr = strtok(NULL, "+DATA: BYTES FROM ()");

            i++;
        }

        printf("+DATA: %s\n", out0);
    }
    if (dev_b[0] == 'N')
    {
        ptr = strtok(dev_b, "Nodo_,=T:HR:DW:Vcc()");

        i = 0;

        while (ptr != NULL)
        {           
            if (i == 1)
                strcat(out1, ptr); /* copies T */
            if (i == 2)
                strcat(out2, ptr); /* copies HR */
            if (i == 3)
                strcat(out3, ptr); /* copies DW */
            if (i == 4)
                strcat(out4, ptr); /* copies Vcc */

            ptr = strtok(NULL, "Nodo_,=T:HR:DW:Vcc()");

            i++;
        }

        printf("Nodo_8: %s, %s, %s, %s\n", out1, out2, out3, out4);
    }

    return 0;
}

With this code I get the following output (with all three cases for 'Nodo'):

$ ./a.out 
+DATA: 043
Nodo_8: 21.45, 45.65, 9.34, 3.46

$ ./a.out 
+DATA: 043
Nodo_8: 21.67, 42.45, 9.23, 3.46

$ ./a.out 
+DATA: 043
Nodo_8: 21.23, 42.65, 9.45, 3.46

'043' corresponds to the '+DATA' part and the rest for the 'Nodo' bit.

Do I have the correct values? I made an assumption from your post. I've tested with all three 'Nodo' samples you've provided.

Answer 2

probably, i think buff contains the hidden characters that you don't expect.

So, I think I need to add a process to get rid of hidden characters from the buff.

E.g.

#include <ctype.h>

void strip_nonprint(char s[]) {
    char *from, *to;
    from = to = s;
    while(*from){
        if(isprint(*from))//note: remove newline
            *to++ = *from++;
        else
            ++from;
    }
    *to = '\0';
}

change code:

n=read(fd,buff,83);
strip_nonprint(buff);//add

/* { i | 2, 4, 5, 6, 7} */
if (i == 2)
 strcat(dev_a, ptr); // copies device
if (i == 4)
 strcat(temp_a, ptr); // copies T
if (i == 5)
   strcat(hr_a, ptr); // copies HR
if (i == 6)
   strcat(dw_a, ptr); // copies DW
if (i == 7)
   strcat(vcc_a, ptr);

Answer 3

As unwind said, the problem is with the strtok(). I think your code made some undefined behaviour because you used the delimiter parameter wrong.

The tokens strtok will find will be: 43,0000,0000,043,8,21.45,45.65,9.34,3.46 I dont even know why was it working for you to extract T,HR, etc.

If You remove the delimiters You gave to it multiple times and You count the tokens again, there will be a working(not elegant at all) solution.

Check the code and the output HERE

Answer 4

I believe you're mis-understanding the second argument to strtok(); it's not a whole delimiter string, it's a "set of characters". In other words, each character in the string is considered a valid separator.

See the manual page for more details, note that it says:

The delim argument specifies a set of bytes that delimit the tokens in the parsed string.

In general, this looks like something you should solve with a plain sscanf(), no need to use strtok() which is a bit more low-level and tricky.