CODEWITHSUNDEEP
c++tokenizeifstream
Miek
Miek

Reputation: 1137

Using isstringstream as a tokenizer in a loop

I would like some help understanding how to deal with isstringstream objects.

I am trying to tokenize each line of a file so I can re-write it in another format after checking certain data values in the tokens. I am loading each line in a tokenVector and iterating through the vector. My code works, but what concerns me is that I have to instantiate a isstringstrem object for each iteration otherwise it does not work. That does not feel right. Her is my code:

std::string line;//each file line
std::ifstream myFile (info.txt.c_str());
if(myFile.is_open()){

     getline(myFile, line);
     std::vector<std::string> tokenVector;

    //create a isstringstream object for tokenizing each line of the file
    std::istringstream hasTokens(line);

    while(hasTokens)
    {
        std::string substring;
        if(! getline(hasTokens, substring,','))
            break;
        tokenVector.push_back(substring);

    }

    //look for some known header names for validation
    if(!tokenVector.empty()){

    if(!(tokenVector[0]=="Time")&&(tokenVector[1] == "Group")&&(tokenVector[2]=="Perception")&&(tokenVector[3] == "Sign")){
        setErrorMesssage("Invalid Header in myFile");
        return false;
        }

        tokenVector.clear();
    }

    //clear the isstringstream object
    hasTokens.str(std::string());

//if header validates, do rest of file

         while(myFile.good()){

            getline(myFile , line);

            //break line into tokens using istringstream
             std::istringstream hasTokens(line);

            //reload the vector of tokens for each line
            while(hasTokens)
            {
                std::string substring;
                if(! getline(hasTokens, substring,','))
                    break;
                tokenVector.push_back(substring);

            }

             otherFileWritingFunction(tokenVector[0], tokenVector[2], tokenVector[4]);    

             tokenVector.clear();
             hasTokens.str(std::string());

        }//end while
    }//end if is_open

This code works, but its not correct because I should only have to instantiate isstringstream once (I think). If I try "hasTokens.str(line)" for each iteration using just the original instantiation of hasTokens, as some example have suggested, it does not work, so I would really appreciate a suggestion.

Thanks

Upvotes: 0

Views: 3816

Answers (2)

Kerrek SB
Kerrek SB

Reputation: 477160

Nope, your worries are misplaced. Create a new stream object when you need it, and dispose of it when you're done. That's the spirit of C++. An object for each purpose, and a purpose for each object (misquoting Frank Herbert). There's nothing "expensive" about constructing a string stream that wouldn't also happen when you reassign the string data of an existing string stream.

Your code is very noisy and redundant, though. The standard idiom goes like this:

std::string line;
while (std::getline(infile, line))
{
    std::istringstream iss(line);

    std::string token; 
    while (iss >> token) { /* do stuff */ }
}

Compressed version (some would call this abuse):

for (std::string line; std::getline(infile, line); )
{
    std::istringstream iss(line);

    for (std::string token; iss >> token; ) { /* ... */ }
}

Upvotes: 6

AJG85
AJG85

Reputation: 16197

The second std::istringstream declaration has an entirely different scope and is being constructed in each iteration so hasTokens.str(std::string()); has no effect.

You could reuse the same object if you did hasTokens.str(line) in the while loop instead.

Upvotes: 1

Related Questions