How to use vImageMatrixMultiply in Swift 3?

Question

I am going to apply a matrix to a CGImage using Accelerate framework.

First, I converted a CGImage to vImage_Buffer,

let bitmapInfo: CGBitmapInfo = [ .floatComponents,
                                 CGBitmapInfo( rawValue: CGImageAlphaInfo.none.rawValue ),
                                 .byteOrder32Little ]

var format = vImage_CGImageFormat( bitsPerComponent: 32,
                                   bitsPerPixel: 32 * 3,
                                   colorSpace: nil,
                                   bitmapInfo: FloatBitmapInfo,
                                   version: 0,
                                   decode: nil,
                                   renderingIntent: .defaultIntent )

var inputBuffer = vImage_Buffer()

vImageBuffer_InitWithCGImage( &inputBuffer,
                              &format,
                              nil,
                              aCGImage,             // input CGImage
                              UInt32( kvImageNoFlags ) )

then tried to apply vImageMatrixMultiply.

vImageMatrixMultiply_PlanarF( &inputBuffer,         // srcs
                              &outputBuffer,        // dests
                              3,                    // src_planes
                              3,                    // dest_planes
                              aMatrix,              // applying matrix
                              &preBias,             // pre_bias
                              &postBias,            // post_bias
                              UInt32( kvImageNoFlags ) )

But vImageMatrixMultiply doesn't accept vImage_Buffer as srcs and dests arguments, and I got a compilation error describing:

Cannot convert value of type "vImage_Buffer" to expected argument type "UnsafePointer< vImage_Buffer >?"

I searched about converting "vImage_Buffer" to "UnsafePointer< vImage_Buffer >?" but haven't found any answer.

So I would like to know how to convert "vImage_Buffer" to "UnsafePointer< vImage_Buffer >?", or how to directly create "UnsafePointer< vImage_Buffer >?" from CGImage, or any other ways to appropriately use vImageMatrixMultiply.

Answer 1

I haven't used vImageMatrixMultiply_PlanarF, but according to the header docs and the function signature, you need to pass pointers to multiple vImage_Buffers for the first two parameters, srcs and dests.

When you need to pass multiple values through UnsafeMutablePointer<T>, you usually declare a variable of [T], and pass it as an inout argument.

In your case T=UnsafePointer<vImage_Buffer>?, so you need to declare a variable of type [UnsafePointer<vImage_Buffer>?]. And each pointer contained in the Array needs to point an appropriately prepared vImage_Buffer.

UPDATED The old code in this answer was using with... methods in a bad manner. Please try with the updated code.

---

Thus, you may need to write something like this:

    //Prepare 3 vImage_Buffers for source planes
    //(vImageConvert_RGBFFFtoPlanarF)
    //You are responsible for filling out the height, width, and rowBytes fields of this structure, and for allocating a data buffer of the appropriate size.
    let inputBuffers: [vImage_Buffer] = (0...2).map {_ in
        var imageBuf = vImage_Buffer()
        imageBuf.width = inputBuffer.width
        imageBuf.height = inputBuffer.height
        imageBuf.rowBytes = Int(imageBuf.width) * MemoryLayout<Float>.size
        imageBuf.data = malloc(imageBuf.rowBytes * Int(imageBuf.height))
        return imageBuf
    }

    //Prepare 3 vImage_Buffers for destination planes
    //(vImageMatrixMultiply_PlanarF)
    //You are responsible for filling out the height, width, and rowBytes fields of these structures, and for allocating data buffers of the appropriate size.
    let outputBuffers: [vImage_Buffer] = (0...2).map {_ in
        var imageBuf = vImage_Buffer()
        imageBuf.width = inputBuffer.width
        imageBuf.height = inputBuffer.height
        imageBuf.rowBytes = Int(imageBuf.width) * MemoryLayout<Float>.size
        imageBuf.data = malloc(imageBuf.rowBytes * Int(imageBuf.height))
        return imageBuf
    }

    //`aMatrix` needs to hold `src_planes`x`dest_planes` elements
    let aMatrix: [Float] = [0.5, 0.25, 0.25,  0.25, 0.5, 0.25, 0.25, 0.25, 0.5]
    var preBias: Float = 0.0
    var postBias: Float = 0.0
    //You need to use pointers or BufferPointers inside the closure, should not take them out of the closure
    inputBuffers.withUnsafeBufferPointer {inputBuffersBP in
        outputBuffers.withUnsafeBufferPointer {outputBuffersBP in
            //Prepare pointer array pointing each vImage_Buffer
            var inputBufferPointers: [UnsafePointer<vImage_Buffer>?] =
                inputBuffers.withUnsafeBufferPointer {inputBuffersBP in
                    (0...2).map{inputBuffersBP.baseAddress! + $0}
            }
            //If you want to use PlanarF format, you need to split RGB into distict vImage_Buffers
            vImageConvert_RGBFFFtoPlanarF(
                &inputBuffer,
                inputBufferPointers[0]!,
                inputBufferPointers[1]!,
                inputBufferPointers[2]!,
                UInt32(kvImageNoFlags)
            )
            //Prepare pointer array pointing each vImage_Buffer
            var outputBufferPointers: [UnsafePointer<vImage_Buffer>?] =
                outputBuffers.withUnsafeBufferPointer {outputBuffersBP in
                    (0...2).map{outputBuffersBP.baseAddress! + $0}
            }
            //All these things prepared properly, you can call `vImageMatrixMultiply_PlanarF` like this...
            vImageMatrixMultiply_PlanarF(
                &inputBufferPointers,         // srcs
                &outputBufferPointers,        // dests
                3,                    // src_planes
                3,                    // dest_planes
                aMatrix,              // applying matrix
                &preBias,             // pre_bias
                &postBias,            // post_bias
                UInt32(kvImageNoFlags)
            )
        }
    }