Gaussian filter with OpenGL Shaders

Question

I am trying to learn Shaders to implement something in my iPhone app. So far I have understood easy examples like making a color image to gray scale, thresholding, etc. Most of the examples involve simple operations in where processing input image pixel I(x,y) results in a simple modification of the colors of the same pixel

But, how about Convolutions?. For example, the easiest example would the Gaussian filter,

in where output image pixel O(x,y) depends not only on I(x,y) but also on surrounding 8 pixels.

O(x,y) = (I(x,y)+ surrounding 8 pixels values)/9;

Normally, this cannot be done with one single image buffer or input pixels will change as the filter is performed. How can I do this with shaders? Also, should I handle the borders myself? or there is a built-it function or something that check invalid pixel access like I(-1,-1) ?

Thanks in advance

PS: I will be generous(read:give a lot of points) ;)

score 11 · Answer 1 · answered Mar 06 '12 at 21:48

A highly optimized shader-based approach for performing a nine-hit Gaussian blur was presented by Daniel Rákos. His process uses the underlying interpolation provided by texture filtering in hardware to perform a nine-hit filter using only five texture reads per pass. This is also split into separate horizontal and vertical passes to further reduce the number of texture reads required.

I rolled an implementation of this, tuned for OpenGL ES and the iOS GPUs, into my image processing framework (under the GPUImageFastBlurFilter class). In my tests, it can perform a single blur pass of a 640x480 frame in 2.0 ms on an iPhone 4, which is pretty fast.

I used the following vertex shader:

 attribute vec4 position;
 attribute vec2 inputTextureCoordinate;

 uniform mediump float texelWidthOffset; 
 uniform mediump float texelHeightOffset; 

 varying mediump vec2 centerTextureCoordinate;
 varying mediump vec2 oneStepLeftTextureCoordinate;
 varying mediump vec2 twoStepsLeftTextureCoordinate;
 varying mediump vec2 oneStepRightTextureCoordinate;
 varying mediump vec2 twoStepsRightTextureCoordinate;

 void main()
 {
     gl_Position = position;

     vec2 firstOffset = vec2(1.3846153846 * texelWidthOffset, 1.3846153846 * texelHeightOffset);
     vec2 secondOffset = vec2(3.2307692308 * texelWidthOffset, 3.2307692308 * texelHeightOffset);

     centerTextureCoordinate = inputTextureCoordinate;
     oneStepLeftTextureCoordinate = inputTextureCoordinate - firstOffset;
     twoStepsLeftTextureCoordinate = inputTextureCoordinate - secondOffset;
     oneStepRightTextureCoordinate = inputTextureCoordinate + firstOffset;
     twoStepsRightTextureCoordinate = inputTextureCoordinate + secondOffset;
 }

and the following fragment shader:

 precision highp float;

 uniform sampler2D inputImageTexture;

 varying mediump vec2 centerTextureCoordinate;
 varying mediump vec2 oneStepLeftTextureCoordinate;
 varying mediump vec2 twoStepsLeftTextureCoordinate;
 varying mediump vec2 oneStepRightTextureCoordinate;
 varying mediump vec2 twoStepsRightTextureCoordinate;

// const float weight[3] = float[]( 0.2270270270, 0.3162162162, 0.0702702703 );

 void main()
 {
     lowp vec3 fragmentColor = texture2D(inputImageTexture, centerTextureCoordinate).rgb * 0.2270270270;
     fragmentColor += texture2D(inputImageTexture, oneStepLeftTextureCoordinate).rgb * 0.3162162162;
     fragmentColor += texture2D(inputImageTexture, oneStepRightTextureCoordinate).rgb * 0.3162162162;
     fragmentColor += texture2D(inputImageTexture, twoStepsLeftTextureCoordinate).rgb * 0.0702702703;
     fragmentColor += texture2D(inputImageTexture, twoStepsRightTextureCoordinate).rgb * 0.0702702703;

     gl_FragColor = vec4(fragmentColor, 1.0);
 }

to perform this. The two passes can be achieved by sending a 0 value for the texelWidthOffset (for the vertical pass), and then feeding that result into a run where you give a 0 value for the texelHeightOffset (for the horizontal pass).

I also have some more advanced examples of convolutions in the above-linked framework, including Sobel edge detection.

Using advantage of linear interpolation is not that new. Where is the benefit of the usual easier way? Lookup at coordinates [-4, -2, 0, 2, 4] to get 10 pixel range and then apply the weighting of each two texels? — djmj, Mar 06 '12 at 22:30
@djmj - I'm not sure that I follow you. What easier way do you mean? What are the two texels that you are referring to? Do you have a reference you can point to that describes this approach? — Brad Larson, Mar 07 '12 at 16:39
See: http://drilian.com/journal/images/TextureGrid.png. If you sample at (2, 0.5) you will get the bilinear interpolated value of pixel (1, 0) and (2, 0). Sampling at (2, 1) you will get the average value of the 4 neighbour pixels! So just use a 2 pixel offset and then apply the gaussian weighting. — djmj, Mar 07 '12 at 17:15
Great answer. I have one question: Rákos calculated the surrounding coordinates in the fragment shader. Is there a reason you calculated the surrounding coordinated in the vertex shader instead? — GingerBreadMane, Sep 27 '12 at 20:31
@GingerBreadMane - For the PowerVR tile-based deferred renderers in the iOS devices (the subject of the question here), calculating this in the vertex shader avoids dependent texture reads. This provides a huge performance boost on these devices, due to the way that they cache texture fetches. — Brad Larson, Sep 28 '12 at 13:21

score 2 · Answer 2 · answered Mar 07 '12 at 17:25

Horizontal Blur using advantage of bilinear interpolation. Vertical blur pass is analog. Unroll to optimise.

//5 offsets for 10 pixel sampling!
float[5] offset = [-4.0f, -2.0f, 0.0f, 2.0f, 4.0f];
//int[5] weight = [1, 4, 6, 4, 1]; //sum = 16
float[5] weightInverse = [0.0625f, 0.25f, 0.375, 0.25f, 0.0625f];

vec4 finalColor = vec4(0.0f);

for(int i = 0; i < 5; i++)
    finalColor += texture2D(inputImage, vec2(offset[i], 0.5f)) * weightInverse[i];

Gaussian filter with OpenGL Shaders

2 Answers2

Linked

Related