Render pixel buffer with Cocoa

Question

I've searched through various Apple's docs and StackOverflow answers, but nothing really helped, still have a blank app's window. I'm trying to display the content of a pixel buffer in the NSWindow, to do that I've allocated a buffer:

UInt8* row = (UInt8 *) malloc(WINDOW_WIDTH * WINDOW_HEIGHT * bytes_per_pixel);

UInt32 pitch = (WINDOW_WIDTH * bytes_per_pixel);

// For each row
for (UInt32 y = 0; y < WINDOW_HEIGHT; ++y) {
  Pixel* pixel = (Pixel *) row;
  // For each pixel in a row
  for (UInt32 x = 0; x < WINDOW_WIDTH; ++x) {
    *pixel++ = 0xFF000000;
  }
  row += pitch;
}

This should prepare a buffer with red pixels. Then I'm creating a NSBitmapImageRep:

NSBitmapImageRep *imageRep = [[NSBitmapImageRep alloc] initWithBitmapDataPlanes:(u8 *) row
                                                                             pixelsWide:WINDOW_WIDTH
                                                                             pixelsHigh:WINDOW_HEIGHT
                                                                          bitsPerSample:8
                                                                        samplesPerPixel:4
                                                                               hasAlpha:YES
                                                                               isPlanar:NO
                                                                         colorSpaceName:NSDeviceRGBColorSpace
                                                                            bytesPerRow:WINDOW_WIDTH * 4
                                                                           bitsPerPixel:32];

Which then converted into NSImage:

NSSize imageSize = NSMakeSize(CGImageGetWidth([imageRep CGImage]), CGImageGetHeight([imageRep CGImage]));
NSImage *image = [[NSImage alloc] initWithSize:imageSize];
[image addRepresentation:imageRep];

Then I'm configuring the view:

NSView *view = [window contentView];
[view setWantsLayer: YES];
[[view layer] setContents: image];

Sadly this doesn't give me the result I expect.

Answer 1

Here are some problems with your code:

1. You are incrementing row by pitch at the end of each y-loop. You never saved the pointer to the beginning of the buffer. When you create your NSBitmapImageRep, you pass a pointer that is past the end of the buffer.

2. You are passing row as the first (planes) argument of initWithBitmapDataPlanes:..., but you need to pass &row. The documentation says

   An array of character pointers, each of which points to a buffer containing raw image data.[…]

   An “array of character pointers” means (in C) you pass a pointer to a pointer.

3. You say “This should prepare a buffer with red pixels.” But you filled the buffer with 0xFF000000, and you said hasAlpha:YES. Depending on the byte order used by the initializer, either you have set the alpha channel to 0, or you have set the alpha channel to 0xFF but set all of the color channels to 0.

   As it happens, you have set each pixel to opaque black (alpha = 0xFF, colors all zero). Try setting each pixel to 0xFF00007F and you'll get a dimmed red (alpha = 0xFF, red = 0x7F).

Thus:

typedef struct {
    uint8_t red;
    uint8_t green;
    uint8_t blue;
    uint8_t alpha;
} Pixel;

@implementation AppDelegate

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
    size_t width = self.window.contentView.bounds.size.width;
    size_t height = self.window.contentView.bounds.size.height;

    Pixel color = { .red=127, .green=0, .blue=0, .alpha=255 };

    size_t pitch = width * sizeof(Pixel);
    uint8_t *buffer = malloc(pitch * height);

    for (size_t y = 0; y < height; ++y) {
        Pixel *row = (Pixel *)(buffer + y * pitch);
        for (size_t x = 0; x < width; ++x) {
            row[x] = color;
        }
    }

    NSBitmapImageRep *rep = [[NSBitmapImageRep alloc] initWithBitmapDataPlanes:&buffer
        pixelsWide:width pixelsHigh:height
        bitsPerSample:8 samplesPerPixel:4 hasAlpha:YES isPlanar:NO
        colorSpaceName:NSDeviceRGBColorSpace
        bytesPerRow:pitch bitsPerPixel:sizeof(Pixel) * 8];
    NSImage *image = [[NSImage alloc] initWithSize:NSMakeSize(width, height)];
    [image addRepresentation:rep];
    self.window.contentView.wantsLayer = YES;
    self.window.contentView.layer.contents = image;
}

@end

Result:

Note that I didn't free buffer. If you free buffer before rep is destroyed, things will go wrong. For example, if you just add free(buffer) to the end of applicationDidFinishLaunching:, the window appears gray.

This is a thorny problem to solve. If you use Core Graphics instead, the memory management is all handled properly. You can ask Core Graphics to allocate the buffer for you (by passing NULL instead of a valid pointer), and it will free the buffer when appropriate.

You have to release the Core Graphics objects you create to avoid memory leaks, but you can do that as soon as you're done with them. The Product > Analyze command can also help you find leaks of Core Graphics objects, but will not help you find leaks of un-freed malloc blocks.

Here's what a Core Graphics solution looks like:

typedef struct {
    uint8_t red;
    uint8_t green;
    uint8_t blue;
    uint8_t alpha;
} Pixel;

@implementation AppDelegate

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
    size_t width = self.window.contentView.bounds.size.width;
    size_t height = self.window.contentView.bounds.size.height;

    CGColorSpaceRef rgb = CGColorSpaceCreateWithName(kCGColorSpaceLinearSRGB);
    CGContextRef gc = CGBitmapContextCreate(NULL, width, height, 8, 0, rgb, kCGImageByteOrder32Big | kCGImageAlphaPremultipliedLast);
    CGColorSpaceRelease(rgb);

    size_t pitch = CGBitmapContextGetBytesPerRow(gc);
    uint8_t *buffer = CGBitmapContextGetData(gc);

    Pixel color = { .red=127, .green=0, .blue=0, .alpha=255 };

    for (size_t y = 0; y < height; ++y) {
        Pixel *row = (Pixel *)(buffer + y * pitch);
        for (size_t x = 0; x < width; ++x) {
            row[x] = color;
        }
    }

    CGImageRef image = CGBitmapContextCreateImage(gc);
    CGContextRelease(gc);
    self.window.contentView.wantsLayer = YES;
    self.window.contentView.layer.contents = (__bridge id)image;
    CGImageRelease(image);
}

@end

Answer 2

Note sure what's going on, but here's code that has been working for years:

static NSImage* NewImageFromRGBA( const UInt8* rawRGBA, NSInteger width, NSInteger height )
{
    size_t rawRGBASize = height*width*4/* sizeof(RGBA) = 4 */;
    // Create a bitmap representation, allowing NSBitmapImageRep to allocate its own data buffer
    NSBitmapImageRep* imageRep = [[NSBitmapImageRep alloc] initWithBitmapDataPlanes:NULL
                                                                         pixelsWide:width
                                                                         pixelsHigh:height
                                                                      bitsPerSample:8
                                                                    samplesPerPixel:4
                                                                           hasAlpha:YES
                                                                           isPlanar:NO
                                                                     colorSpaceName:NSCalibratedRGBColorSpace
                                                                        bytesPerRow:0
                                                                       bitsPerPixel:0];
    NSCAssert(imageRep!=nil,@"failed to create NSBitmapImageRep");
    NSCAssert((size_t)[imageRep bytesPerPlane]==rawRGBASize,@"alignment or size of CGContext buffer and NSImageRep do not agree");
    // Copy the raw bitmap image into the new image representation
    memcpy([imageRep bitmapData],rawRGBA,rawRGBASize);
    // Create an empty NSImage then add the bitmap representation to it
    NSImage* image = [[NSImage alloc] initWithSize:NSMakeSize(width,height)];
    [image addRepresentation:imageRep];

    return image;
}