Error when writing a very simple HLSL pixel shader

Question

I am writing I SlimDX application to test color contrast sensitivity thresholds of the human visual system. The subject is presented with a large letter that is at low contrast with with the background and asked to identify the letter. However I need the ability to display more colors then is available with a simple 8bit per channel color depth. (i.e. rgb values 0-255). The way I plan to due this is through a simple "dithering" algorithm implemented as an HLSL pixel shader. Basically if request slimDX to render the text to a surface with a color of Color4( 0.55f, 0.55f, 0.55f ) which corresponds to a rgb value of ( 140.25 , 140.25 , 140.25 ) I want each color channel of each pixel to have a 25% chance of being set to 141 and a 75% chance of getting set to 140. This should (in the limit of many pixels) lead to a letter that appears as a shade of gray one 1/4 of the way in-between 140 and 141.
I am however getting errors in my prototype shader code when trying to compile with fxc. I am getting an Illegal character code when I compile and I don't know why. Also if you are a HLSL guru please look over my code and make any comments that strike you. Note I got the HLSL random function from an answer to this question. Can I generate a random number inside a pixel shader?

Below is my shader code. Please forgive me if there a many mistakes as its my first HLSL code:

float4 DitherByChance(float2 coords : TEXCOORD0) : COLOR0
{
    float4 newColor;    // The pixel color to return
    float4 oldColor = tex2D(s0, coords);

    // Note I know that as the code stands now rCutOff = gCutOff = bCutOff I will sort a fix for this out later
    float rCutOff = random(coords);     // A random float that determines if the red channel will be rounded up or down
    float gCutOff = random(coords);     // A random float that determines if the green channel will be rounded up or down
    float bCutOff = random(coords);     // A random float that determines if the blue channel will be rounded up or down

    float rPercentChance = frac(oldColor.r * 255);  //Chance to round red channel up
    float gPercentChance = frac(oldColor.g * 255);  //Chance to round green channel up
    float bPercentChance = frac(oldColor.b * 255);  //Chance to round blue channel up

    //In the code below (1/255) is the floating point represntaion of an incress of one on the 0-255 RGB scale

    if (rCutOff <= rPercentChance) newColor.r = oldColor.r + ((1 - rPercentChance) / 255);  //Bump up one r value
    else newColor.r = oldColor.r - rPercentChance * (1 / 255);                  //Bump down to ensure r is not rounded up

    if (gCutOff <= gPercentChance) newColor.g = oldColor.g + ((1 - gPercentChance) / 255);  //Bump up one g value
    else newColor.b = oldColor.b - bPercentChance * (1 / 255);                  //Bump down to ensure g is not rounded up

    if (bCutOff <= bPercentChance) newColor.b = oldColor.b + ((1 - bPercentChance) / 255);  //Bump up one b value
    else newColor.b = oldColor.b - bPercentChance * (1 / 255);                  //Bump down to ensure b is not rounded up

    return newColor;
}

// Input: It uses texture coords as the random number seed.
// Output: Random number: [0,1), that is between 0.0 and 0.999999... inclusive.
// Author: Michael Pohoreski
// Copyright: Copyleft 2012 :-)
float random( vec2 p )
{
  // We need irrationals for pseudo randomness.
  // Most (all?) known transcendental numbers will (generally) work.
  const vec2 r = vec2(
    23.1406926327792690,  // e^pi (Gelfond's constant)
     2.6651441426902251); // 2^sqrt(2) (Gelfond–Schneider constant)
  return fract( cos( mod( 123456789., 1e-7 + 256. * dot(p,r) ) ) );  
}

EDIT: With the help of the answer below I have come a little closer to a solution but am still having some issues. I have taken catflier's comments that the random function I borrowed is written in GLSL not HLSL. I had to change vec2 to float2 and put the random function at the top. In addition to that I also had to write a mod function because GLSL mod() != HLSL fmod. And the GLSL fract() was changed to the HLSL frac(). I also had a problem where my text file was encoded with UTF-8 which I guess causes some issues with fxc. After re-encoding to ANSI I was able to get some better error messages. I am still having some issues however.

I tried compiling using fxc /Zi /E DitherByChance DitherByChance.fx as well as the fxc /Zi /E DitherByChance /LD DitherByChance.fx because I am targeting DX9 but I the following error both times:

undeclared indentifier 's0' error from this line float4 oldColor = tex2D(s0, coords);

I am not sure what this s0 argument is exactly I just saw it in another shader example. I saw another example where someone had a Global sampler2D variable declared and used that instead of the s0 so I tried this:

sampler2D Tex0;
...
.... 
.....
float4 oldColor = tex2D(Tex0, coords); 

However now when I run fxc /Zi /E DitherByChance /LD DitherByChance.fx I get the following error:

vs_2_0 target does not support texture lookup

Any ideas?

Answer 1

First you need to replace vec2 by float2 in your random function, since vec2 is glsl syntax.

Then your random function should be placed above your PixelShader (c-style).

Your shader should now compile, if not, please post error message.

Edit from latest bits:

sampler2D Tex0;

Is a sampler unit, you need to bind it to your texture, since sampler will tell how to filter it (linear/point), and address it (wrap/clamp...)

http://msdn.microsoft.com/en-us/library/windows/desktop/bb509644(v=vs.85).aspx

Since you compile a pixel shader, you need to use fxc /T ps_3_0, since you compile as pixel shader, not a vertex shader.