Odd characters received over serial ☐[J

Question

I've created a terminal emulator using a library, connect to a serial device with another library and this returns data to my terminal. I can see in the log that every time I write a character in the terminal and sent it over serial, the following three characters are returned along with the correct one. ☐[J. When writing to the terminal these characters do now show up. They are handled in this code in some way but I'm not sure which part, perhaps doEscRightSquareBracket:

 private void process(byte b, boolean doUTF8) {
        // Let the UTF-8 decoder try to handle it if we're in UTF-8 mode
        if (doUTF8 && mUTF8Mode && handleUTF8Sequence(b)) {
            return;
        }

        // Handle C1 control characters
        if ((b & 0x80) == 0x80 && (b & 0x7f) <= 0x1f) {
            /* ESC ((code & 0x7f) + 0x40) is the two-byte escape sequence
               corresponding to a particular C1 code */
            process((byte) 27, false);
            process((byte) ((b & 0x7f) + 0x40), false);
            return;
        }

        switch (b) {
        case 0: // NUL
            // Do nothing
            break;

        case 7: // BEL
            /* If in an OSC sequence, BEL may terminate a string; otherwise do
             * nothing */
            if (mEscapeState == ESC_RIGHT_SQUARE_BRACKET) {
                doEscRightSquareBracket(b);
            }
            break;

        case 8: // BS
            setCursorCol(Math.max(0, mCursorCol - 1));
            break;

        case 9: // HT
            // Move to next tab stop, but not past edge of screen
            setCursorCol(nextTabStop(mCursorCol));
            break;

        case 13:
            setCursorCol(0);
            break;

        case 10: // CR
        case 11: // VT
        case 12: // LF
            doLinefeed();
            break;

        case 14: // SO:
            setAltCharSet(true);
            break;

        case 15: // SI:
            setAltCharSet(false);
            break;


        case 24: // CAN
        case 26: // SUB
            if (mEscapeState != ESC_NONE) {
                mEscapeState = ESC_NONE;
                emit((byte) 127);
            }
            break;

        case 27: // ESC
            // Starts an escape sequence unless we're parsing a string
            if (mEscapeState != ESC_RIGHT_SQUARE_BRACKET) {
                startEscapeSequence(ESC);
            } else {
                doEscRightSquareBracket(b);
            }
            break;

        default:
            mContinueSequence = false;
            switch (mEscapeState) {
            case ESC_NONE:
                if (b >= 32) {
                    emit(b);
                }
                break;

            case ESC:
                doEsc(b);
                break;

            case ESC_POUND:
                doEscPound(b);
                break;

            case ESC_SELECT_LEFT_PAREN:
                doEscSelectLeftParen(b);
                break;

            case ESC_SELECT_RIGHT_PAREN:
                doEscSelectRightParen(b);
                break;

            case ESC_LEFT_SQUARE_BRACKET:
                doEscLeftSquareBracket(b); // CSI
                break;

            case ESC_LEFT_SQUARE_BRACKET_QUESTION_MARK:
                doEscLSBQuest(b); // CSI ?
                break;

            case ESC_PERCENT:
                doEscPercent(b);
                break;

            case ESC_RIGHT_SQUARE_BRACKET:
                doEscRightSquareBracket(b);
                break;

            case ESC_RIGHT_SQUARE_BRACKET_ESC:
                doEscRightSquareBracketEsc(b);
                break;

            default:
                unknownSequence(b);
                break;
            }
            if (!mContinueSequence) {
                mEscapeState = ESC_NONE;
            }
            break;
        }
    }

This is not a problem with the terminal as it is filtering it. But now I want to write the returned data into an editText and the odd characters are being written. What are they, how do I stop them? They must be some normal case of something that can happen as the terminal filters them out? You can see it here when I am typing exit and it should be mirrored on the right:

Answer 1

Esc-[-J is an ANSI escape code, it instructs the terminal to clear the screen from the cursor down. The problem with filtering them out is that among these commands many change how the visible text is constructed: they can move the cursor, erase parts, etc. So, just filtering them out may not give the desired result. But in this case it just seems like a precaution to make sure that the area where you normally type is cleared.

What I consider the best solution, but quite probably overkill in your case, is to integrate a VT100 interpreter in your program (for example this one) that converts a stream of text and command codes into a memory view of the screen, and save that memory. It will be a perfect representation of what the sending program would like to see on the screen at any time.