parsing a string using strtok - twice in C++

Question

I want to parse a given string using strtok - but using it twice. String to parse: "x=2;y=30". Desired output: An array of strings containing [x,y] and an array of string containing [2,30]. Meaning, I want to first use ";" as a delimiter, then using "=" as a delimiter. How can I do that - in cpp?

Answer 1

In C++, I wouldn't use std::strtok(). std::string provides nice find methods in multiple flavors, e.g. std::string::find_first_of().

As I couldn't resist to fiddle with the ancient strtok(), a little demo for all that:

#include <cassert>
#include <cstring>
#include <iostream>
#include <vector>

int main()
{
  char input[] = "x=2;y=30";
  std::vector<char*> leftHandSides, rightHandSides;
  for (char *in = input;;) {
    char *lhs = strtok(in, "=");
    in = nullptr;
    if (!lhs) break;
    char *rhs = strtok(in, ";");
    if (!rhs) break;
    leftHandSides.push_back(lhs);
    rightHandSides.push_back(rhs);
  }
  for (size_t i = 0, n = rightHandSides.size(); i < n; ++i) {
    std::cout << leftHandSides[i] << '=' << rightHandSides[i] << '\n';
  }
}

Output:

x=2
y=30

Live Demo on coliru

Notes:

1. strtok() has to be called with a pointer to buffer (to tokenize) for the first time, and with a 0-pointer for further results (of same buffer). Hence, the char *in = input; which is overridden after char *lhs = strtok(in, "="); with in = nullptr;. It is important that this is done in the first iteration. That it happens in all other iterations as well is tolerated as it doesn't cause harm. (I could've checked with an additional if but hadn't won anything with this.)

2. strtok() modifies the input string. Hence, I used char input[] = "x=2;y=30"; intentional.
   (With const char *input = "x=2;y=30";, I would've made code with U.B.)

---

Finally, a variant using std::istringstream and std::getline() instead of std::strtok():

#include <iostream>
#include <sstream>
#include <string>
#include <vector>

int main()
{
  std::string input = "x=2;y=30";
  std::vector<std::string> leftHandSides, rightHandSides;
  for (std::istringstream in(input);;) {
    std::string lhs, rhs;
    if (!std::getline(in, lhs, '=')
      || !std::getline(in, rhs, ';')) break;
    leftHandSides.push_back(lhs);
    rightHandSides.push_back(rhs);
  }
  for (size_t i = 0, n = rightHandSides.size(); i < n; ++i) {
    std::cout << leftHandSides[i] << '=' << rightHandSides[i] << '\n';
  }
}

Output:

x=2
y=30

Live Demo on coliru