NSTokenField's tokenField:completionsForSubstring:indexOfToken:indexOfSelectedItem: indexOfToken always zero

Question

I've tried a simple first example of NSTokenField with a document-based ARC-using application, making my Document class an NSTokenFieldDelegate. It works but for one thing: the delegate method tokenField:completionsForSubstring:indexOfToken:indexOfSelectedItem: never sees anything but 0 for indexOfToken, even when I successfully edit a token which isn't the first in the string of tokens. I'm using XCode 4.5 on OS X 10.8.2 with the 10.8 framework.

Question: why always 0? I expect it to be the index of the token in the indirectly-seen array of tokens 0 .. n - 1 in the field being edited by the user.

To reproduce, start a project as above and add the text below, then use the XIB editor and drag an NSTokenField onto the document window, set the token field as the document's tokenField and make the document instance the delegate of the token field.

Document.h:

#import <Cocoa/Cocoa.h>

@interface Document : NSDocument <NSTokenFieldDelegate>
{
    IBOutlet NSTokenField *tokenField;  // of (Token *).
    NSMutableDictionary *tokens;        // of (Token *).
}
@end

Token.h:

#import <Foundation/Foundation.h>

@interface Token : NSObject
@property (strong, nonatomic) NSString *spelling;
- (id)initWithSpelling:(NSString *)s;
@end

Token.m:

#import "Token.h"

@implementation Token
@synthesize spelling;

- (id)initWithSpelling:(NSString *)s
{
    self = [super init];
    if (self)
        spelling = s;
    return self;
}

@end

Document.m:

#import "Document.h"
#import "Token.h"

@implementation Document

- (id)init
{
    self = [super init];
    if (self) {
        tokens = [NSMutableDictionary dictionary];
    }
    return self;
}

...

#pragma mark NSTokenFieldDelegate methods

- (NSArray *)tokenField:(NSTokenField *)tokenField
completionsForSubstring:(NSString *)substring
           indexOfToken:(NSInteger)tokenIndex
    indexOfSelectedItem:(NSInteger *)selectedIndex
{
    NSLog(@"tokenField:completionsForSubstring:\"%@\" indexOfToken:%ld indexOfSelectedItem:",
          substring, tokenIndex);
    NSMutableArray *result = [NSMutableArray array];
    for (NSString *key in tokens) {
        //NSLog(@"match? \"%@\"", key);
        if ([key hasPrefix:substring])
            [result addObject:key];
    }
    return result;
}

- (id)tokenField:(NSTokenField *)tokenField representedObjectForEditingString:(NSString *)editingString
{
    NSLog(@"tokenField:representedObjectForEditingString:\"%@\"", editingString);
    Token *token;
    if ((token = [tokens objectForKey:editingString]) == nil) {
        token = [[Token alloc] initWithSpelling:editingString];
        [tokens setObject:token forKey:editingString];
        //NSLog(@"token %@", [token description]);
        NSLog(@"tokens %@", [tokens description]);
    }
    return token;
}

- (NSString *)tokenField:(NSTokenField *)tokenField displayStringForRepresentedObject:(id)representedObject
{
    NSString *spelling = [representedObject spelling];
    NSLog(@"tokenField:displayStringForRepresentedObject: = \"%@\"", spelling);
    return spelling;
}

@end

Entry of tokens is terminated with a newline or comma character.

Answer 1

Here is my solution in swift which is working fine for me. Just add this to your viewController.

     func tokenIndex() -> Int {
        var index = 0
        let range = self.tokenField?.currentEditor()?.selectedRange
        let rangeLocation = range?.location ?? 0

        let string = self.tokenField?.currentEditor()?.string as NSString?
        if let subString = string?.substring(to: rangeLocation) as NSString? {
            let maxLimit = subString.length

            for i in 0..<maxLimit {
                //each token is represented as NSAttachmentCharacter,
                //so count it till current selected range
                if subString.character(at: i) == unichar(NSAttachmentCharacter) {
                    index += 1
                }
            }
        }

        return index
    }

Answer 2

This definitely seems like something you should report to Apple.

To determine the index of the edited token in an NSTokenField, I first made a subclass of NSTextView so that I have a custom field editor. (If you go this route, don’t forget to set up your NSTextView instance to be a field editor with -[NSTextView setFieldEditor:].) Then, I subclassed NSTokenFieldCell, overriding -[NSCell fieldEditorForView:] to

1. create an instance of my NSTextView subclass,
2. set this NSTextView instance to be self’s delegate, and
3. return this NSTextView instance.

Implement tokenFieldCell:completionsForSubstring:indexOfToken:indexOfSelectedItem: in your NSTextView subclass:

- (NSArray *)tokenFieldCell:(NSTokenFieldCell *)tokenFieldCell completionsForSubstring:(NSString *)substring indexOfToken:(NSInteger)tokenIndex indexOfSelectedItem:(NSInteger *)selectedIndex
{
    // The tokenIndex passed to this function seems to be 0 in all cases, so we
    // need to determine the tokenIndex ourselves. The range returned by
    // NSText's selectedRange method treats non-plain-text tokens as if they
    // have unit length. So, for each token, subtract from the range location
    // either the token's length if it's a plain text token, or 1 if it's any
    // other style of token. Each time we subtract from the range location,
    // increment tokenIndex. When the range location becomes less than or equal
    // to 0, tokenIndex will be the index of the edited token.
    tokenIndex = 0;
    NSInteger rangeLocation = self.selectedRange.location;
    for (id token in tokenFieldCell.objectValue) {
        if ([self tokenFieldCell:tokenFieldCell styleForRepresentedObject:token] == NSPlainTextTokenStyle) {
            rangeLocation -= [self tokenFieldCell:tokenFieldCell displayStringForRepresentedObject:token].length;
        } else {
            rangeLocation--;
        }
        if (rangeLocation > 0) {
            tokenIndex++;
        } else {
            break;
        }
    }
}

The idea here is to use the fact that the selectedRange of an NSTextView is calculated assuming that non–plain-text tokens have a length of 1. By subtracting lengths of tokens from the selectedRange location until the location is negative, we can determine the token index.

Note that your NSTextView subclass must also implement tokenFieldCell:displayStringForRepresentedObject: and tokenFieldCell:styleForRepresentedObject: for this to work.