syntax highlight richedit control not working poperly

Question

I am trying to implement a syntax highlight editor with a richedit, it has works well with the current selected line, but I may be missing something. The CRichEdit is my own wrapper implementation of the richedit controller, the problem seems that the text is not selected properly even though I made sure the selected range generated with the code were what I get with the EM_EXGETSEL message. The selection seems increasing 1 as the lines goes down, so I decided to ed_source.sendMessage(EM_LINEFROMCHAR, pos, 0) to the range which partially fix the problem except for some few lines where the coloring seems some time one or to position before and the real appropriate, so that is why I thing I may not be understanding something.

void parse(WIN::CRichEdit &ed_source, bool curseline)
{
    int pos, offset = 0;
    char delimiter[]={" \n\r(){};"}, *tok, *start;
    CStringA s;
    CString text;
    CWnd api;

    if(curseline){      
        ed_source.getLine(ed_source.getRow() - 1, text);
        offset = ed_source.sendMessage(EM_LINEINDEX, -1, 0);
    }else{
        text = ed_source.getCaption();
    }

    s = text;
    start = s.c_str();
    if(!start) return;

    tok = strtok(s.c_str(), delimiter);

    CHARRANGE cr = ed_source.getSelecteRange();
    ed_source.sendMessage(EM_HIDESELECTION, 1, 0) ;
    CHARRANGE range;
    while(tok)
    {
        int len = strlen(tok);

        pos = (tok - start);
        int x = ed_source.sendMessage(EM_LINEFROMCHAR, pos, 0);
        range.cpMin = offset + pos - x;
        range.cpMax = range.cpMin + len;

        ed_source.selectRange(range);
        if(isReserved(tok)){

            ed_source.setTextStyle(true, false);
            ed_source.setTextColor(keyboardColor);
        }else
            if(isType(tok)){
                ed_source.setTextStyle(false, false);
                ed_source.setTextColor(typeColor);
            }else {
                ed_source.setTextStyle(false, true);
                ed_source.setTextColor(textColor);
            }
        tok = strtok(0, delimiter);
    }

    ed_source.sendMessage(EM_HIDESELECTION, 0, 0) ;
    ed_source.selectRange(cr);
}

just to be more specific the moment I call the above function is immediately after loading the text on it. I assumed you may want to see the implementation of some of the above functions so here they are.

CHARRANGE CRichEdit::getSelecteRange()
{
    CHARRANGE crg = {0} ;
    sendMessage(EM_EXGETSEL, 0, (LPARAM)&crg);
    return crg;
}

void CRichEdit::selectRange(const CHARRANGE &cr)
{
    sendMessage( EM_EXSETSEL, 0, (LPARAM) &cr);
}


void CRichEdit::setTextColor(COLORREF col)
{ 
    CHARFORMAT format;
    memset(&format, 0, sizeof(CHARFORMAT));
    format.cbSize       = sizeof(CHARFORMAT);
    format.dwMask       = CFM_COLOR;
    format.crTextColor  = col;

    sendMessage( EM_SETCHARFORMAT, SCF_SELECTION, (LPARAM) &format);
}

Answer 1

Have a look at this article for some ideas:

Faster rich edit syntax highlighting http://bcbjournal.org/articles/vol3/9910/Faster_rich_edit_syntax_highlighting.htm

It was written for the TRichEdit control in C++Builder, but the majority of the tips in it use straight Win32 API calls, and the few places that use VCL idioms can be easily adapted into Win32 equivilents.

Update: Try removing EM_LINEFROMCHAR from your loop. offset + pos is already an absolute character position within the RichEdit, there should be no need to adjust it on each loop iteration. If you really want to take line indexes into account then you should be looping through the lines one row at a time parsing each row individually, not parsing the entire content as a single string. Try something more like this instead:

void parse(WIN::CRichEdit &ed_source, bool curseline)
{
    int startLine, endLine, offset;
    const char* delimiters = " \n\r(){};";
    char *tok, *start;
    CStringA s;
    CWnd api;

    if (curseline)
    {
        startLine = ed_source.getRow() - 1;
        endLine = startLine + 1;
    }
    else
    {
        startLine = 0;
        endLine = ed_source.sendMessage(EM_GETLINECOUNT, 0, 0);
    }

    CHARRANGE cr = ed_source.getSelecteRange();

    int eventMask = ed_source.SendMessage(EM_SETEVENTMASK, 0, 0);
    ed_source.SendMessage(WM_SETREDRAW, FALSE, 0);

    for (int line = startLine; line < endLine; ++line)
    {
        CString text;
        ed_source.getLine(line, text);

        s = text;
        start = s.c_str();
        if (!start) continue;

        offset = ed_source.sendMessage(EM_LINEINDEX, line, 0);

        tok = strtok(start, delimiters);
        while (tok)
        {
            CHARRANGE range;
            range.cpMin = offset + (int)(tok - start);
            range.cpMax = range.cpMin + strlen(tok);

            ed_source.selectRange(range);
            if (isReserved(tok))
            {
                ed_source.setTextStyle(true, false);
                ed_source.setTextColor(keyboardColor);
            }
            else if (isType(tok))
            {
                ed_source.setTextStyle(false, false);
                ed_source.setTextColor(typeColor);
            }
            else
            {
                ed_source.setTextStyle(false, true);
                ed_source.setTextColor(textColor);
            }

            tok = strtok(0, delimiters);
        }
    }

    ed_source.SendMessage(WM_SETREDRAW, TRUE, 0);
    ed_source.Invalidate(); // whatever your wrapper does to call ::InvalidateRect()

    ed_source.SendMessage(EM_SETEVENTMASK, 0, eventMask);

    ed_source.selectRange(cr);
}

With that said, instead of using getLine() and strtok() to parse text, you might consider using EM_FINDWORDBREAK to locate words and EM_EXSETSEL/EM_GETSELTEXT to retreive the characters of each word. That way, you are using less memory and letting the RichEdit do more of the searching for you. You can use EM_SETWORDBREAKPROC/EX if you want to customize the word delimiters searched for.