CODEWITHSUNDEEP
swiftunsafe-pointerschromakey
impression7vx
impression7vx

Reputation: 1863

Data via Buffer Post Xcode 11.5 Update

What I Have:

Referencing Apple's Chroma Key Code, it states that we can create a Chroma Key Filter Cube via

func chromaKeyFilter(fromHue: CGFloat, toHue: CGFloat) -> CIFilter?
{
    // 1
    let size = 64
    var cubeRGB = [Float]()
        
    // 2
    for z in 0 ..< size {
        let blue = CGFloat(z) / CGFloat(size-1)
        for y in 0 ..< size {
            let green = CGFloat(y) / CGFloat(size-1)
            for x in 0 ..< size {
                let red = CGFloat(x) / CGFloat(size-1)
                    
                // 3
                let hue = getHue(red: red, green: green, blue: blue)
                let alpha: CGFloat = (hue >= fromHue && hue <= toHue) ? 0: 1
                    
                // 4
                cubeRGB.append(Float(red * alpha))
                cubeRGB.append(Float(green * alpha))
                cubeRGB.append(Float(blue * alpha))
                cubeRGB.append(Float(alpha))
            }
        }
    }

    let data = Data(buffer: UnsafeBufferPointer(start: &cubeRGB, count: cubeRGB.count))

    // 5
    let colorCubeFilter = CIFilter(name: "CIColorCube", withInputParameters: ["inputCubeDimension": size, "inputCubeData": data])
    return colorCubeFilter
}

I then created a function to be able to insert any image into this filter and return the filtered image.

public func filteredImage(ciimage: CIImage) -> CIImage? {
    let filter = chromaKeyFilter(fromHue: 110/360, toHue: 130/360)! //green screen effect colors
    filter.setValue(ciimage, forKey: kCIInputImageKey)
    return RealtimeDepthMaskViewController.filter.outputImage
}

I can then execute this function on any image and obtain a chroma key'd image.

if let maskedImage = filteredImage(ciimage: ciimage) {
    //Do something
}
else {
    print("Not filtered image")
}

Update Issues:

let data = Data(buffer: UnsafeBufferPointer(start: &cubeRGB, count: cubeRGB.count))

However, once I updated Xcode to v11.6, I obtain the warning Initialization of 'UnsafeBufferPointer<Float>' results in a dangling buffer pointer as well as a runtime error Thread 1: EXC_BAD_ACCESS (code=1, address=0x13c600020) on the line of code above.

I tried addressing this issue with this answer to correct Swift's new UnsafeBufferPointer warning. The warning is then corrected and I no longer have a runtime error.

Problem

Now, although the warning doesn't appear and I don't experience a runtime error, I still get the print statement Not filtered image. I assume that the issue stems from the way the data is being handled, or deleted, not entirely sure how to correctly handle UnsafeBufferPointers alongside Data.

What is the appropriate way to correctly obtain the Data for the Chroma Key?

Upvotes: 2

Views: 425

Answers (1)

lentil
lentil

Reputation: 694

I wasn't sure what RealtimeDepthMaskViewController was in this context, so just returned the filter output instead. Apologies if this was meant to be left as-is. Also added a guard statement with the possibility of returning nil - which matches your optional return type for the function.

public func filteredImage(ciImage: CIImage) -> CIImage? {
    guard let filter = chromaKeyFilter(fromHue: 110/360, toHue: 130/360) else { return nil }
    filter.setValue(ciImage, forKey: "inputImage")
    return filter.outputImage // instead of RealtimeDepthMaskViewController.filter.outputImage
}

For the dangling pointer compiler warning, I found a couple approaches:

// approach #1
var data = Data()
cubeRGB.withUnsafeBufferPointer { ptr in
    data = Data(buffer: ptr)
}

// approach #2
let byteCount = MemoryLayout<Float>.size * cubeRGB.count
let data = Data(bytes: &cubeRGB, count: byteCount)

One caveat: looked at this with Xcode 11.6 rather than 11.5

Upvotes: 5

Related Questions