FreeType OpenGL dynamic Text = abysmal performance

Question

I'm currently searching for bottlenecks in my code and it turns out the GUI is one of them. Well, not actually the GUI but rather the dynamic text that is drawn there.

Initialization

        if (FT_Init_FreeType(&m_FreeType))
            throw Helpers::ExceptionWithMsg("Could not init freetype lib");

        if (FT_New_Face(m_FreeType, "res\\fonts\\FreeSans.ttf", 0, &m_FontFace))
            throw Helpers::ExceptionWithMsg("Could not open font");

        m_ShaderID = ... // Loads the corresponding shader
        m_TextColorLocation = glGetUniformLocation(m_ShaderID, "color");
        m_CoordinatesLocation = glGetAttribLocation(m_ShaderID, "coord");

        glGenBuffers(1, &m_VBO);

        FT_Set_Pixel_Sizes(m_FontFace, 0, m_FontSize);
        glyph = m_FontFace->glyph;

        glGenTextures(1, &m_Texture);

        glActiveTexture(GL_TEXTURE0);
        glBindTexture(GL_TEXTURE_2D, m_Texture);

        // We require 1 byte alignment when uploading texture data 
        glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

        // Linear filtering usually looks best for text 
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

        // Clamping to edges is important to prevent artifacts when scaling 
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

        glUseProgram(m_ShaderID);
            glUniform4f(m_TextColorLocation, m_TextColor.x, m_TextColor.y, m_TextColor.z, m_TextColor.w);
        glUseProgram(0);

What I do: I initialize FreeType, get the Font, initialize the shader and all uniforms.

Then I create the vbo for the textureCoordinates, set the Pixels for the font, get the glyph.

Now I generate the texture, activate it, bind it... I want to set all the parameters and then the uniform that never changes.

Rendering:

    glUseProgram(m_ShaderID);

    glEnable(GL_BLEND);
    glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

    glActiveTexture(GL_TEXTURE0);
    glBindTexture(GL_TEXTURE_2D, m_Texture);

    // Linear filtering usually looks best for text 
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

    // Set up the VBO for our vertex data 
    glEnableVertexAttribArray(m_CoordinatesLocation);
    glBindBuffer(GL_ARRAY_BUFFER, m_VBO);
    glVertexAttribPointer(m_CoordinatesLocation, 4, GL_FLOAT, GL_FALSE, 0, 0);

    GLfloat cursorPosX = m_X;
    GLfloat cursorPosY = m_Y;
    for (size_t i = 0; i < m_Text.size(); ++i)
    {
        // If Loading a char fails, just continue
        if (FT_Load_Char(m_FontFace, m_Text[i], FT_LOAD_RENDER))
            continue;

        glTexImage2D(GL_TEXTURE_2D, 0, GL_ALPHA, glyph->bitmap.width, glyph->bitmap.rows, 0, GL_ALPHA, GL_UNSIGNED_BYTE, glyph->bitmap.buffer);

        // Calculate the vertex and texture coordinates 
        GLfloat x2 = cursorPosX + glyph->bitmap_left * m_SX;
        GLfloat y2 = -cursorPosY - glyph->bitmap_top * m_SY;
        GLfloat w = glyph->bitmap.width * m_SX;
        GLfloat h = glyph->bitmap.rows * m_SY;

        PointStruct box[4] =
        {
            { x2, -y2, 0, 0 },
            { x2 + w, -y2, 1, 0 },
            { x2, -y2 - h, 0, 1 },
            { x2 + w, -y2 - h, 1, 1 }
        };

        // Draw the character on the screen
        glBufferData(GL_ARRAY_BUFFER, sizeof box, box, GL_DYNAMIC_DRAW);
        glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

        // Advance the cursor to the start of the next character 
        cursorPosX += glyph->advance.x / 64 * m_SX;
        cursorPosY += glyph->advance.y / 64 * m_SY;
    }

    glDisableVertexAttribArray(m_CoordinatesLocation);
    glDeleteTextures(1, &m_Texture);
    glDisable(GL_BLEND);
    glUseProgram(0);

Setting the shader and stuff is obvious.

For each render call I activate the texture, bind it, enable the VBO I store my textureCoordinates in. Then I iterate over every character in the text load it with FT_LOAD_CHAR. I then specify the texture with glTexImage2D, calculate vertex and texture coordinates and draw everything.

That seems to be highly inefficient, but I find no way to improve the performance and yet have readable text.

I wanted to set the text parameters just once in the init -> all chars are boxes.

I wanted to set GL_DYNAMIC_DRAW to GL_STATIC_DRAW... not much difference. What else can I do?

The text I render is dynamic, it changes (or may change) each frame, so I'm kind of stuck.

I query the performance of this stuff with a query. If I do not render the dynamic text it's very low, but if I render the dynamic text it gets up very high... there is not much else going on in this pass, it's just drawing the GUI.

What really bothers me

One thing I really don't understand (may be the sunny day...)

If I do not set linear filtering in the render-method() I get strange cube-glyphs, but why is that? OpenGL is a state machine, the texture-parameters are set to the one currently bound. So if I set the Min and Mag filter to GL_LINEAR in the initialization why isn't that enough?

If I remove those 2 lines in the render I get way better performance from the query (much lower numbers), but it doesn't drawn anything readable.

Answer 1

There are two easy options to improve performance.

1. Create a single texture that has all the characters that you will need to render at set offsets. Your text string then becomes a model (buffer object) that will reference the correct series of offsets to create your text string. This has the disadvantage that you will not be able to do kerning or any other fancy font joining.
2. Use Pango and Cairo to fully render your text as a single bitmap. This bitmap will have the text formatted with correct kerning and joining. You then only upload and draw a single texture for your entire text output.

Answer 2

This is absolutely going to be slow.

For each render call I activate the texture, bind it, enable the VBO I store my textureCoordinates in. Then I iterate over every character in the text load it with FT_LOAD_CHAR. I then specify the texture with glTexImage2D, calculate vertex and texture coordinates and draw everything.

The problem, unfortunately, is hard. Here is the method I use:

• There is one texture, with the GL_RED8 format, which stores glyphs.

• Whenever a new glyph is needed, it is added to the texture. This is done by calling FT_Render_Glyph() and copying the result into the texture buffer. If the new glyph doesn't fit, the whole glyph texture is resized and repacked. (I use the skyline algorithm for packing glyphs since it's simple.)

• If any new glyphs have been added, then I call glTexSubImage2D(). The code should be structured so that this is only called once per frame.

• To render text, I create a VBO that contains vertex and texture coordinates for all the quads necessary to render a piece of text. (Please understand that "quad" means two triangles, not GL_QUAD).

So, when you change what text you want to render,

• You have to update the VBO, but only once per frame

• You might have to update the texture, but only once per frame, and this will probably happen less frequently as the glyph texture fills up with the glyphs you use.

A good way to prototype this kind of system is to render all of the glyphs in a font into the texture at first, but this doesn't work well if you end up using multiple fonts and styles, or if you want to render Chinese, Korean, or Japanese text.

Additional considerations are line breaking, glyph substitution, kerning, bidi, general problems with international text, how to specify styling, et cetera. I recommend using HarfBuzz in combination with FreeType. HarfBuzz handles difficult glyph substitution and positioning issues. None of this is strictly necessary if your program has English text only.

There are some libraries that do all of this, but I have not used them.

An alternative method, if you want to cut the gordian knot, is to embed a web browser like Chromium (Awesomium, WebKit, Gecko—many choices) in your application, and farm out all text rendering to that.

Answer 3

your bottleneck are probably the many draw calls. firstly you buffer the texture inside of your draw routine: instead provide a texture where you can map characters onto quad-positions and then replace the following:

    // Calculate the vertex and texture coordinates 
    GLfloat x2 = cursorPosX + glyph->bitmap_left * m_SX;
    GLfloat y2 = -cursorPosY - glyph->bitmap_top * m_SY;
    GLfloat w = glyph->bitmap.width * m_SX;
    GLfloat h = glyph->bitmap.rows * m_SY;

    PointStruct box[4] =
    {
        { x2, -y2, 0, 0 },
        { x2 + w, -y2, 1, 0 },
        { x2, -y2 - h, 0, 1 },
        { x2 + w, -y2 - h, 1, 1 }
    };

    // Draw the character on the screen
    glBufferData(GL_ARRAY_BUFFER, sizeof box, box, GL_DYNAMIC_DRAW);
    glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

with code that pre-processes your text, produces a larger PointStruct buffer (don't forget to adjust your texture coordinates for the look-up-texture) and draws multiple characters per draw call.