CODEWITHSUNDEEP
iosswiftcoreml
markwalberg
markwalberg

Reputation: 321

EXC_BAD_ACCESS (code=1) when updating UIImage of UIImageView

I am new with iOS and CoreML. I have a very simple UI with two UIImageViews (one should be the input, and the second should be the output). When tapping the first one, the image should be processed by a neural network and the output should be displayed in the second one.

However, when I try to download the image from the MLMultiArray output object and create an UIImage from it which I can then upload to the second UIImageView I get an EXC_BAD_ACCESS (code=1) .

I have reduced the problem to not calling the neural network processing at all, just trying to create a new image from a MLMultiArray. The outcome was the same.

After that I tried generating an UIImage from an empty buffer. The image is created correctly, but if I attempt to update the UIImageView to use it, I get the same error.

If I try to update the second UIImageView to a different image (e.g.: the input image) everything works fine.

I assume this is a memory management issue about the UIImage object I am creating but I am not able to figure out what I am doing wrong

class ViewController: UIViewController {

    @IBOutlet weak var out: UIImageView!

    override func viewDidLoad() {
        super.viewDidLoad()
    }

    @IBAction func imageTapped(_ sender: UITapGestureRecognizer) {
        let imageView = sender.view as? UIImageView

        if let imageToAnalyse = imageView?.image {
            if let outImg = process(forImage: imageToAnalyse) {
                out.image = outImg
            }
        }
    }

    func process (forImage inImage:UIImage) -> UIImage? {

        let size = CGSize(width: 512, height: 512)
        let mlOut = try? MLMultiArray(shape: [1, size.height, size.width] as [NSNumber], dataType: .float32)
        let newImage = getSinglePlaneImage(inBuffer: mlOut!, width: Int(size.width), height: Int(size.height))

        return newImage
    }

    func getSinglePlaneImage(inBuffer: MLMultiArray, width: Int, height: Int) -> UIImage
    {
        var newImage: UIImage

//        let floatPtr =  inBuffer.dataPointer.bindMemory(to: Float32.self, capacity: inBuffer.count)
//        let floatBuffer = UnsafeBufferPointer(start: floatPtr, count: inBuffer.count)
//        let pixelValues : [UInt8]? = Array(floatBuffer).map({UInt8( ImageProcessor.clamp( (($0) + 1.0) * 128.0, minValue: 0.0, maxValue: 255.0) ) })

        //simulating pixels from MLMultiArray 
        let pixels : [UInt8]? = Array(repeating: 0, count: 512*512)

        var imageRef: CGImage?

        if var pixelValues = pixels {
            let bitsPerComponent = 8
            let bytesPerPixel = 1
            let bitsPerPixel = bytesPerPixel * bitsPerComponent
            let bytesPerRow = bytesPerPixel * width
            let totalBytes = height * bytesPerRow

            imageRef = withUnsafePointer(to: &pixelValues, {
                ptr -> CGImage? in
                var imageRef: CGImage?
                let colorSpaceRef = CGColorSpaceCreateDeviceGray()
                let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.none.rawValue).union(CGBitmapInfo())
                let data = UnsafeRawPointer(ptr.pointee).assumingMemoryBound(to: UInt8.self)
                let releaseData: CGDataProviderReleaseDataCallback = {
                    (info: UnsafeMutableRawPointer?, data: UnsafeRawPointer, size: Int) -> () in
                }

                if let providerRef = CGDataProvider(dataInfo: nil, data: data, size: totalBytes, releaseData: releaseData) {
                    imageRef = CGImage(width: width,
                                       height: height,
                                       bitsPerComponent: bitsPerComponent,
                                       bitsPerPixel: bitsPerPixel,
                                       bytesPerRow: bytesPerRow,
                                       space: colorSpaceRef,
                                       bitmapInfo: bitmapInfo,
                                       provider: providerRef,
                                       decode: nil,
                                       shouldInterpolate: false,
                                       intent: CGColorRenderingIntent.defaultIntent)
                }


                return imageRef
            })
        }

        newImage = UIImage(cgImage: imageRef!)

        return newImage
    }
}

Upvotes: 0

Views: 704

Answers (1)

OOPer
OOPer

Reputation: 47876

Seems your code would convert 512x512-float32 array into 512x512-UInt8 Array successfully, so I write this answer based on the uncommented version of your code. (Though, the conversion is not efficient enough and has some room to improve.)

UPDATE

The following description is not the right solution for the OP's issue. Just kept for record. Please skip to UPDATED CODE at the bottom of this answer.

OLD CODE (NOT the right solution)

First of all, the worst flaw in the code are the following two lines:

imageRef = withUnsafePointer(to: &pixelValues, {

    let data = UnsafeRawPointer(ptr.pointee).assumingMemoryBound(to: UInt8.self)

The first line above passes a pointer to [UInt8]?, in Swift, [UInt8]? (aka Optional<Array<UInt8>>) is an 8-byte struct, not a contiguous region like C-arrays.

The second is more dangerous. ptr.pointee is [UInt8]?, but accessing Swift Arrays through pointer is not guaranteed. And passing an Array to UnsafeRawPointer.init(_:) may create a temporal region which would be deallocated just after the call to the initializer.

As you know, accessing a dangling pointer does not make harm in some limited condition occasionally, but may generate unexpected result at any time.

I would write something like this:

func getSinglePlaneImage(inBuffer: MLMultiArray, width: Int, height: Int) -> UIImage {

    //simulating pixels from MLMultiArray
    //...
    let pixelValues: [UInt8] = Array(repeating: 0, count: 1*512*512)

    let bitsPerComponent = 8
    let bytesPerPixel = 1
    let bitsPerPixel = bytesPerPixel * 8
    let bytesPerRow = bytesPerPixel * width
    let totalBytes = height * bytesPerRow

    let imageRef = pixelValues.withUnsafeBytes({bytes -> CGImage? in
        var imageRef: CGImage?
        let colorSpaceRef = CGColorSpaceCreateDeviceGray()
        let bitmapInfo: CGBitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.none.rawValue)

        let data = bytes.baseAddress!.assumingMemoryBound(to: UInt8.self)
        let releaseData: CGDataProviderReleaseDataCallback = {_,_,_ in }

        if let providerRef = CGDataProvider(dataInfo: nil, data: data, size: totalBytes, releaseData: releaseData) {
            imageRef = CGImage(width: width,
                               height: height,
                               bitsPerComponent: bitsPerComponent,
                               bitsPerPixel: bitsPerPixel,
                               bytesPerRow: bytesPerRow,
                               space: colorSpaceRef,
                               bitmapInfo: bitmapInfo,
                               provider: providerRef,
                               decode: nil,
                               shouldInterpolate: false,
                               intent: .defaultIntent)
        }


        return imageRef
    })

    let newImage = UIImage(cgImage: imageRef!)

    return newImage
}

When you want a pointer pointing to the starting element of an Array, use withUnsafeBytes and use the pointer (in fact, it is an UnsafeRawBufferPointer) inside the closure argument.

One more, your pixels or pixelValues have no need to be an Optional.

Or else, you can create a grey-scale image with Float32 for each pixel.

func getSinglePlaneImage(inBuffer: MLMultiArray, width: Int, height: Int) -> UIImage {

    //simulating pixels from MLMultiArray
    //...
    let pixelValues: [Float32] = Array(repeating: 0, count: 1*512*512)

    let bitsPerComponent = 32 //<-
    let bytesPerPixel = 4 //<-
    let bitsPerPixel = bytesPerPixel * 8
    let bytesPerRow = bytesPerPixel * width
    let totalBytes = height * bytesPerRow

    let imageRef = pixelValues.withUnsafeBytes({bytes -> CGImage? in
        var imageRef: CGImage?
        let colorSpaceRef = CGColorSpaceCreateDeviceGray()
        let bitmapInfo: CGBitmapInfo = [CGBitmapInfo(rawValue: CGImageAlphaInfo.none.rawValue),
                          .byteOrder32Little, .floatComponents] //<-
        let data = bytes.baseAddress!.assumingMemoryBound(to: Float32.self)
        let releaseData: CGDataProviderReleaseDataCallback = {_,_,_ in }

        if let providerRef = CGDataProvider(dataInfo: nil, data: data, size: totalBytes, releaseData: releaseData) {
            imageRef = CGImage(width: width,
                               height: height,
                               bitsPerComponent: bitsPerComponent,
                               bitsPerPixel: bitsPerPixel,
                               bytesPerRow: bytesPerRow,
                               space: colorSpaceRef,
                               bitmapInfo: bitmapInfo,
                               provider: providerRef,
                               decode: nil,
                               shouldInterpolate: false,
                               intent: CGColorRenderingIntent.defaultIntent)
        }


        return imageRef
    })

    let newImage = UIImage(cgImage: imageRef!)

    return newImage
}

Both work as expected in my testing project, but if you find something wrong, please inform me.

UPDATED CODE (Hope this is the right solution)

I was missing the fact that CGDataProvider keeps the pointer when created with init(dataInfo:data:size:releaseData:) even after a CGImage is created. So, it may be referenced after the closure to withUnsafeBytes is finished, when it is not valid.

You should better use CGDataProvider.init(data:) in such cases.

func getSinglePlaneImage(inBuffer: MLMultiArray, width: Int, height: Int) -> UIImage {
    var newImage: UIImage

    //let floatPtr =  inBuffer.dataPointer.bindMemory(to: Float32.self, capacity: inBuffer.count)
    //let floatBuffer = UnsafeBufferPointer(start: floatPtr, count: inBuffer.count)
    //let pixelValues: Data = Data((floatBuffer.lazy.map{
    //    UInt8(ImageProcessor.clamp((($0) + 1.0) * 128.0, minValue: 0.0, maxValue: 255.0))
    //})

    //simulating pixels from MLMultiArray
    //...
    let pixelValues = Data(count: 1*512*512) // <- ###

    var imageRef: CGImage?

    let bitsPerComponent = 8
    let bytesPerPixel = 1
    let bitsPerPixel = bytesPerPixel * bitsPerComponent
    let bytesPerRow = bytesPerPixel * width

    let colorSpaceRef = CGColorSpaceCreateDeviceGray()
    let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.none.rawValue)

    if let providerRef = CGDataProvider(data: pixelValues as CFData) { // <-###
        imageRef = CGImage(width: width,
                           height: height,
                           bitsPerComponent: bitsPerComponent,
                           bitsPerPixel: bitsPerPixel,
                           bytesPerRow: bytesPerRow,
                           space: colorSpaceRef,
                           bitmapInfo: bitmapInfo,
                           provider: providerRef,
                           decode: nil,
                           shouldInterpolate: false,
                           intent: CGColorRenderingIntent.defaultIntent)
    }

    newImage = UIImage(cgImage: imageRef!)

    return newImage
}

As far as I tried, this does not crash even in actual device with number of repeated touches. Please try. Thanks for your patience.

Upvotes: 1

Related Questions