CODEWITHSUNDEEP
swiftarkitcore-imageciimagecvpixelbuffer
RealUglyDuck
RealUglyDuck

Reputation: 377

RGB values of CIImage pixel

I want to access the average colour value of a specific area of CVPixelBuffer that I get from ARFrame in real-time. I managed to crop the image, use filter to calculate average colour and after converting to CGImage I get the value from the pixel but unfortunately, it affects the performance of my app (FPS drops below 30fps). I think that the reason for that is using CGContext. Is there a way to access colour without converting CIImage to CGImage? This is code that I'm using at the moment:

func fun() {
  let croppVector = CIVector(cgRect: inputImageRect)

  guard let filter = CIFilter(
    name: "CIAreaAverage",
    parameters: [kCIInputImageKey: image, kCIInputExtentKey: croppVector]
  ),
    let outputImage = filter.outputImage,
    let cgImage = context.createCGImage(
      outputImage,
      from: CGRect(x: 0, y: 0, width: 1, height: 1)
    ),
    let dataProvider = cgImage.dataProvider,
    let data = CFDataGetBytePtr(dataProvider.data) else { return nil }

  let color = UIColor(
    red: CGFloat(data[0]) / 255,
    green: CGFloat(data[1]) / 255,
    blue: CGFloat(data[2]) / 255,
    alpha: CGFloat(data[3]) / 255
  )
}

Upvotes: 4

Views: 2871

Answers (1)

Frank Rupprecht
Frank Rupprecht

Reputation: 10418

I think there is not too much you can do here – reduction operations like this are expensive.

A few things you can try:

  • Set up your CIContext to not perform any color management by setting the .workingColorSpace and .outputColorSpace options to NSNull().
  • Render directly into a piece of memory instead of going through a CGImage. The context has the method render(_ image: CIImage, toBitmap data: UnsafeMutableRawPointer, rowBytes: Int, bounds: CGRect, format: CIFormat, colorSpace: CGColorSpace?) you can use for that. Also pass nil as color space here. You should be able to just pass a "pointer" to a simd_uchar4 var as data here. rowBytes would be 4 and format would be .BGRA8 in this case, I think.
  • You can also try to scale down your image (which already is a reduction operation) before you do the average calculation. It wouldn't be the same value, but a fair approximation – and it might be faster.

Upvotes: 2

Related Questions