CODEWITHSUNDEEP
objective-ccocoastringcore-foundation
qwzybug
qwzybug

Reputation: 553

How to get the token type from a CFStringTokenizer in Cocoa?

The CFStringTokenizer documentation has two conflicting statements in CFStringTokenizerAdvanceToNextToken():

CFStringTokenizerAdvanceToNextToken

...

Return Value

The type of the token if the tokenizer succeeded in finding a token and setting it as current token. Returns kCFStringTokenizerTokenNone if the tokenizer failed to find a token. For possible values, see “Token Types.”

...

If a token is found, it is set as the current token and the function returns true; otherwise the current token is invalidates and the function returns false.

The first paragraph (returning a token type) is what I'd like to see: it lets you, for example, check if a token is made up entirely of non-alphanumeric characters. However, the second paragraph (returning true or false) seems to be what is actually happening.

Any ideas why that would be, or how to work around it?

Upvotes: 1

Views: 854

Answers (3)

pulkitsinghal
pulkitsinghal

Reputation: 4084

If what you're looking for is the ability to ignore CFNumber type tokens, then you may use:

while (tokenType != kCFStringTokenizerTokenNone) {
    if ( (tokenType & kCFStringTokenizerTokenHasHasNumbersMask) == 0) { // ignore numbers
        range = CFStringTokenizerGetCurrentTokenRange(tokenizer);
        NSString *token = [string substringWithRange:NSMakeRange(range.location, range.length)];
        [tokensArray addObject:token];
    }
    tokenType = CFStringTokenizerAdvanceToNextToken(tokenizer);
}

Upvotes: 0

Peter Hosey
Peter Hosey

Reputation: 96333

It's possible they meant “true” and “false” in the broader sense of “non-zero” and “zero”. If it finds a token, the function returns kCFStringTokenizerTokenNormal (which is 1) or some combination of the masks (either way, non-zero/“true”). If it doesn't, the function returns kCFStringTokenizerTokenNone (which is zero/“false”).

It's certainly vague language, though, so please file a documentation bug on the Apple Bug Reporter.

Upvotes: 0

smorgan
smorgan

Reputation: 21579

The header comment doesn't mention returning true or false, and when the header and the online docs disagree it's often the header that is correct.

In a simple test I'm able to get return values other than 0 and 1, so the problem you are seeing may be more specific; can you post sample code that's failing?

Upvotes: 1

Related Questions