CODEWITHSUNDEEP
iosobjective-ccopengl-esgpuimage
imstillalive
imstillalive

Reputation: 369

iOS - How to create custom GPUImageFilters with OpenGLES

I'd like to know how to create custom filters for GPUImage. Right now I can create a sepia custom filter from this code(this was provided by Brad Larson on GitHub as an example):

varying highp vec2 textureCoordinate;

uniform sampler2D inputImageTexture;

void main()
{
    lowp vec4 textureColor = texture2D(inputImageTexture, textureCoordinate);
    lowp vec4 outputColor;
    outputColor.r = (textureColor.r * 0.393) + (textureColor.g * 0.769) + (textureColor.b * 0.189);
    outputColor.g = (textureColor.r * 0.349) + (textureColor.g * 0.686) + (textureColor.b * 0.168);    
    outputColor.b = (textureColor.r * 0.272) + (textureColor.g * 0.534) + (textureColor.b * 0.131);
    outputColor.a = 1.0;

    gl_FragColor = outputColor;
}

I'm curious to know how Brad Larson knew these numbers and what it means. I have searched everywhere and did not find a tutorial on this. Can someone please guide me to create my own custom photo Filters with Photoshop and then translate it into a .fsh code file?

For example, if I am to change an image into a pink tone in Photoshop. How do I get these numbers in the code above?

Upvotes: 0

Views: 681

Answers (1)

Brad Larson
Brad Larson

Reputation: 170317

Your question is a little broad, as you can literally write an entire book on how to create custom shaders, but it's a commonly asked one, so I can at least point people in the right direction.

Filters in GPUImage are written in the OpenGL Shading Language (GLSL). There are slight differences between the OpenGL targets (Mac, desktop Linux) and OpenGL ES ones (iOS, embedded Linux) in that shaders on the latter use precision qualifiers that are missing on the former. Beyond that, the syntax is the same.

Shader programs are composed of a matched pair of a vertex shader and a fragment shader. A vertex shader operates over each vertex and usually handles geometric manipulations. A fragment shader operates over each fragment (pixel, generally) and calculates the color to be output to the screen at that fragment.

GPUImage deals with image processing, so most of the time you'll be working only with fragment shaders and relying on one of the stock vertex shaders. The above is an example of a fragment shader that takes in each pixel of an input texture (the image from the previous step in the processing pipeline), manipulates its color values, and writes out the result to the gl_FragColor builtin.

The first line in the main() function:

lowp vec4 textureColor = texture2D(inputImageTexture, textureCoordinate);

uses the texture2D() function to read the pixel color in the inputImageTexture at a given coordinate (passed in from the vertex shader in the first stage). These coordinates are normalized to the 0.0-1.0 range, and therefore are independent of the input image size.

The values are loaded into a vector type (vec4) that contains multiple components within a single type. In this case, the color components for red, green, blue, and alpha are stored in this four-component vector and can be accessed via .r, .g, .b, and .a. The color component values are also normalized to the 0.0-1.0 range, in case you're used to working with 0-255 values.

In the particular case of a sepia tone filter, I'm attempting to apply a well-known color matrix for a sepia tone effect to convert the incoming color to an outgoing one. That requires matrix multiplication, which I do explicitly in the code above. In the actual framework, this is done as matrix math using builtin types within the shader.

There are many, many ways to manipulate colors to achieve certain effects. The GPUImage framework is full of them, based largely on things like color conversion standards published by Adobe or other organizations. For a given effect, you should identify if one of these existing implementations will do what you want before setting out to write your own.

If you do want to write your own, first figure out the math required to translate incoming colors into whatever output you want. Once you have that, writing the shader code is easy.

The lookup filter in GPUImage takes another approach, and that's to start with a lookup image that you manipulate in Photoshop under the filter conditions you wish to mimic. You then take that filtered lookup image and attach it to the lookup filter. It translates between the incoming colors and their equivalents in the lookup to provide arbitrary color manipulation.

Once you have your shader, you can create a new filter around that in a few different ways. I should say that my new GPUImage 2 framework greatly simplifies that process, if you're willing to forgo some backwards compatibility.

Upvotes: 3

Related Questions