MTLTexture displays UIImages incorrectly if converted to textures

Question

I have this method

- (id<MTLTexture>) createTextureFromImage:(UIImage*) image device:(id<MTLDevice>) device
{
  CGImageRef imageRef = image.CGImage;

  //  size_t width = CGImageGetWidth(imageRef);
  //  size_t height = CGImageGetHeight(imageRef);

  size_t width = self.view.frame.size.width;
  size_t height = self.view.frame.size.height;

  size_t bitsPerComponent = CGImageGetBitsPerComponent(imageRef);
  size_t bitsPerPixel = CGImageGetBitsPerPixel(imageRef);

  CGColorSpaceRef colorSpace = CGImageGetColorSpace(imageRef);
  CGImageAlphaInfo alphaInfo = CGImageGetAlphaInfo(imageRef);

  NSLog(@"%@", colorSpace);

  CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault | alphaInfo;
  NSLog(@"bitmap info %u", bitmapInfo);

  CGContextRef context = CGBitmapContextCreate( NULL, width, height, bitsPerComponent, (bitsPerPixel / 8) * width, colorSpace, bitmapInfo);
  if( !context )
  {
    NSLog(@"Failed to load image, probably an unsupported texture type");
    return nil;
  }

  CGContextDrawImage( context, CGRectMake( 0, 0, width, height ), image.CGImage );

  MTLPixelFormat format = MTLPixelFormatBGRA8Unorm;

  MTLTextureDescriptor *texDesc = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:format
                                                                                     width:width
                                                                                    height:height
                                                                                 mipmapped:NO];
  id<MTLTexture> texture = [device newTextureWithDescriptor:texDesc];

  [texture replaceRegion:MTLRegionMake2D(0, 0, width, height)
             mipmapLevel:0
               withBytes:CGBitmapContextGetData(context)
             bytesPerRow:4 * width];

  return texture;
}

When i pass it an image and use the resulting texture, it gives the overall image a red hue. My pixel format for my MTLRenderPipelineDescriptor is set to MTLPixelFormatBGRA8Unorm, and i get the feeling that the issue is somewhere around here. I've attached a sample of the image to give you an idea of what it should be, and what it's showing up as. I know it's not the image, as if i just add the image straight to the view it works fine, but this could also be because the view uses RGBA8 color values and the metalview is using BGRA8 color values. If i change my pixelFormat to RGBA8 though, my app crashes with the warning

For color attachment 0, the render pipeline's pixelFormat (MTLPixelFormatRGBA8Unorm) does not match the framebuffer's pixelFormat (MTLPixelFormatBGRA8Unorm).

The only thing is that I can't see where i'm setting the framebuffers pixel format.

Thanks!

EDIT - This the processing code that my image goes through once i've split it into a pixel array.

-(UIImage *) script:(NSArray*)pixelArray {

  float width = self.view.frame.size.width;
  float height = self.view.frame.size.height;

  //create drawing context
  UIGraphicsBeginImageContextWithOptions(CGSizeMake(width, height), NO, 0.0f);
  CGContextRef context = UIGraphicsGetCurrentContext();

  int k = 0;
  do {
    for (int y = 0; y < height + 1; y++)
    {
      for (int x = 1; x < width; x++)
      {


        if (k < array.count){
          UIColor* pixelColor = array[k];


          CGContextSetFillColorWithColor(context,  pixelColor.CGColor);
          CGContextFillRect(context, CGRectMake(x, y, 1.0f, 1.0f));
        }
      } else {

        CGContextSetFillColorWithColor(context, [UIColor clearColor].CGColor);
        CGContextFillRect(context, CGRectMake(x, y, 1.0f, 1.0f));
      }
      k++;
    }
    k++;
  }
} while (k < 62791);

//capture resultant image
UIImage *image = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
return image;
}

Answer 1

There are two separate concerns here: matching the pixel format between your render pipeline state and your framebuffer, and matching the pixel format between your image data and the pixel format of the texture you create from it.

In the first case, it's sensible to use MTLPixelFormatBGRA8Unorm. This has to be set on your CAMetalLayer (or MTKView) as well as your render pipeline state's primary color attachment. If they don't agree, you'll get the exception you mentioned. This configuration looks something like this:

metalView.colorPixelFormat = MTLPixelFormatBGRA8Unorm;
// and elsewhere...
pipelineStateDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormatBGRA8Unorm;

In the latter case, what matters is that the component ordering of your image data matches the format you choose for your texture. The colorspace you get from your UIImage is sRGB, so you need to specify MTLPixelFormatRGBA8Unorm as your texture's pixel format (where you currently have MTLPixelFormatBGRA8Unorm in your texture creation code), or swap the bytes in the red and blue channels manually.

The pixel formats of your view and the texture you create from the image don't have to be the same. Metal swizzles colors when you sample textures and when you write to the framebuffer, so within your shaders, you can always assume an RGBA ordering.