Monogame Shader Porting Issues

Question

Ok so I ported a game I have been working on over to Monogame, however I'm having a shader issue now that it's ported. It's an odd bug, since it works on my old XNA project and it also works the first time I use it in the new monogame project, but not after that unless I restart the game.

The shader is a very simple shader that looks at a greyscale image and, based on the grey, picks a color from the lookup texture. Basically I'm using this to randomize a sprite image for an enemy every time a new enemy is placed on the screen. It works for the first time an enemy is spawned, but doesn't work after that, just giving a completely transparent texture (not a null texture).

Also, I'm only targeting Windows Desktop for now, but I am planning to target Mac and Linux at some point.

Here is the shader code itself.

sampler input : register(s0);
Texture2D colorTable;
float seed; //calculate in program, pass to shader (between 0 and 1)

sampler colorTableSampler = 
sampler_state
{
    Texture = <colorTable>;
};

float4 PixelShaderFunction(float2 c: TEXCOORD0) : COLOR0
{
    //get current pixel of the texture (greyscale)
    float4 color = tex2D(input, c);
    //set the values to compare to.
    float hair = 139/255; float hairless = 140/255;
    float shirt = 181/255; float shirtless = 182/255;
    //var to hold new color
    float4 swap;
    //pixel coordinate for lookup
    float2 i;
    i.y = 1;

    //compare and swap
    if (color.r >= hair && color.r <= hairless)
    {
        i.x = ((0.5 + seed + 96)/128);
        swap = tex2D(colorTableSampler,i);
    }
    if (color.r >= shirt && color.r <= shirtless)
    {
        i.x = ((0.5 + seed + 64)/128);
        swap = tex2D(colorTableSampler,i);
    }
    if (color.r == 1)
    {
        i.x = ((0.5 + seed + 32)/128);
        swap = tex2D(colorTableSampler,i);
    }
    if (color.r == 0)
    {
        i.x = ((0.5 + seed)/128);
        swap = tex2D(colorTableSampler, i);
    }

    return swap;
}

technique ColorSwap
{
    pass Pass1
    {
        // TODO: set renderstates here.

        PixelShader = compile ps_2_0 PixelShaderFunction();
    }
}

And here is the function that creates the texture. I should also note that the texture generation works fine without the shader, I just get the greyscale base image.

public static Texture2D createEnemyTexture(GraphicsDevice gd, SpriteBatch sb)
        {
            //get a random number to pass into the shader.
            Random r = new Random();
            float seed = (float)r.Next(0, 32);
            //create the texture to copy color data into
            Texture2D enemyTex = new Texture2D(gd, CHARACTER_SIDE, CHARACTER_SIDE);
            //create a render target to draw a character to.
            RenderTarget2D rendTarget = new RenderTarget2D(gd, CHARACTER_SIDE, CHARACTER_SIDE,
                false, gd.PresentationParameters.BackBufferFormat, DepthFormat.None);
            gd.SetRenderTarget(rendTarget);
            //set background of new render target to transparent.
            //gd.Clear(Microsoft.Xna.Framework.Color.Black);
            //start drawing to the new render target
            sb.Begin(SpriteSortMode.Immediate, BlendState.Opaque,
                    SamplerState.PointClamp, DepthStencilState.None, RasterizerState.CullNone);
            //send the random value to the shader.
            Graphics.GlobalGfx.colorSwapEffect.Parameters["seed"].SetValue(seed);
            //send the palette texture to the shader.            
            Graphics.GlobalGfx.colorSwapEffect.Parameters["colorTable"].SetValue(Graphics.GlobalGfx.palette);
            //apply the effect
            Graphics.GlobalGfx.colorSwapEffect.CurrentTechnique.Passes[0].Apply();
            //draw the texture (now with color!)
            sb.Draw(enemyBase, new Microsoft.Xna.Framework.Vector2(0, 0), Microsoft.Xna.Framework.Color.White);
            //end drawing
            sb.End();
            //reset rendertarget
            gd.SetRenderTarget(null);
            //copy the drawn and colored enemy to a non-volitile texture (instead of render target)
            //create the color array the size of the texture.
            Color[] cs = new Color[CHARACTER_SIDE * CHARACTER_SIDE];
            //get all color data from the render target
            rendTarget.GetData<Color>(cs);
            //move the color data into the texture.
            enemyTex.SetData<Color>(cs);
            //return the finished texture.
            return enemyTex;
        }

And just in case, the code for loading in the shader:

BinaryReader Reader = new BinaryReader(File.Open(@"Content\\shaders\\test.mgfx", FileMode.Open));
colorSwapEffect = new Effect(gd, Reader.ReadBytes((int)Reader.BaseStream.Length));

If anyone has ideas to fix this, I'd really appreciate it, and just let me know if you need other info about the problem.

Answer 1

Change ps_2_0 to ps_4_0_level_9_3.

Monogame cannot use shaders built on HLSL 2.

Also the built in sprite batch shader uses ps_4_0_level_9_3 and vs_4_0_level_9_3, you will get issues if you try to replace the pixel portion of a shader with a different level shader.

This is the only issue I can see with your code.

Answer 2

I don't see anything super obvious just reading through it, but really this could be tricky for someone to figure out just looking at your code.

I'd recommend doing a graphics profile (via visual studio) and capturing the frame which renders correctly then the frame rendering incorrectly and comparing the state of the two.

Eg, is the input texture what you expect it to be, are pixels being output but culled, is the output correct on the render target (in which case the problem could be Get/SetData), etc.

Answer 3

I am not sure why you have "at" (@) sign in front of the string, when you escaped backslash - unless you want to have \\ in your string, but it looks strange in the file path.

You have wrote in your code:

BinaryReader Reader = new BinaryReader(File.Open(@"Content\\shaders\\test.mgfx", FileMode.Open));

Unless you want \\ inside your string do

BinaryReader Reader = new BinaryReader(File.Open(@"Content\shaders\test.mgfx", FileMode.Open));

or

BinaryReader Reader = new BinaryReader(File.Open("Content\\shaders\\test.mgfx", FileMode.Open));

but do not use both.