CODEWITHSUNDEEP
c++c++11boosttokenizeboost-tokenizer
Ufx
Ufx

Reputation: 2695

How to tokenize string by delimiters?

I need to tokenize string by delimiters.

For example:

For "One, Two Three,,, Four" I need to get {"One", "Two", "Three", "Four"}.

I am attempting to use this solultion https://stackoverflow.com/a/55680/1034253

std::vector<std::string> strToArray(const std::string &str,
                                    const std::string &delimiters = " ,")
{
    boost::char_separator<char> sep(delimiters.c_str());
    boost::tokenizer<boost::char_separator<char>> tokens(str.c_str(), sep);

    std::vector<std::string> result;
    for (const auto &token: tokens) {
        result.push_back(token);
    }

    return result;
}

But I get the error:

boost-1_57\boost/tokenizer.hpp(62): error C2228: left of '.begin' must have class/struct/union type is 'const char *const'

Upvotes: 0

Views: 1393

Answers (4)

sanfranciscan
sanfranciscan

Reputation: 11

Short way.

string tmp = "One, Two, Tree, Four";
int pos = 0;
while (pos = tmp.find(", ") and pos > 0){
    string s = tmp.substr(0, pos);
    tmp = tmp.substr(pos+2);
    cout << s;
}

Upvotes: 1

Neil
Neil

Reputation: 485

I see a lot of boost answers, so I thought I'd supply a non-boost answer:

template <typename OutputIter>
void Str2Arr( const std::string &str, const std::string &delim, int start, bool ignoreEmpty, OutputIter iter )
{
    int pos = str.find_first_of( delim, start );
    if (pos != std::string::npos) {
        std::string nStr = str.substr( start, pos - start );
        trim( nStr );

        if (!nStr.empty() || !ignoreEmpty)
            *iter++ = nStr;
        Str2Arr( str, delim, pos + 1, ignoreEmpty, iter );
    }
    else
    {
        std::string nStr = str.substr( start, str.length() - start );
        trim( nStr );

        if (!nStr.empty() || !ignoreEmpty)
          *iter++ = nStr;
    }
}

std::vector<std::string> Str2Arr( const std::string &str, const std::string &delim )
{
    std::vector<std::string> result;
    Str2Arr( str, delim, 0, true, std::back_inserter( result ) );
    return std::move( result );
}

trim can be any trim function, I used this SO answer. It makes use of std::back_inserter and recursion. You could easily do it in a loop, but this sounded like more fun :)

Upvotes: 0

PaulMcKenzie
PaulMcKenzie

Reputation: 35440

Change this:

boost::tokenizer<boost::char_separator<char>> tokens(str.c_str(), sep);

To this:

boost::tokenizer<boost::char_separator<char>> tokens(str, sep);

Link: http://www.boost.org/doc/libs/1_57_0/libs/tokenizer/tokenizer.htm

The container type requires a begin() function, and a const char* (which is what c_str()) returns does not meet this requirement.

Upvotes: 2

Drew Dormann
Drew Dormann

Reputation: 63745

Boost's tokenizer is probably overkill for the task you describe.

boost::split was written for this exact task.

std::vector<std::string> strToArray(const std::string &str,
                                    const std::string &delimiters = " ,")
{
    using namespace boost;
    std::vector<std::string> result;
    split( result, str, is_any_of(delimiters), token_compress_on );
    return result;
}

That optional token_compress_on signifies that your ,,, input shouldn't imply empty string tokens between those commas.

Upvotes: 1

Related Questions