CODEWITHSUNDEEP
iosswiftmacoscoremlapple-vision
Gana
Gana

Reputation: 1027

ObjectDetection: Output different from CreateML vs programatically

I want to extract known objects from an image. I created a ObjectDetector model using CreateML App. When I test with CreateML preview the detection works perfectly fine however via code, something seems to be wrong.

Below is the sample code section that I wrote. I am saving pictures using the boundingbox, however, the predicted images are completely different when I test with CreateML preview. I have tried all options, please let me know what is wrong in my code.

func extractSpecifcSectioninImage(image: NSImage){
    var requests = [VNRequest]()
    var picCount = 1
    let modelURL = Bundle.main.url(forResource: "ObjectDetection", withExtension: "mlmodelc")!
    
    do {
        let visionModel = try VNCoreMLModel(for: MLModel(contentsOf: modelURL))
        let objectRecognition = VNCoreMLRequest(model: visionModel, completionHandler: { (request, error) in
            if let results = request.results {
                for observation in results where observation is VNRecognizedObjectObservation {
                    guard let objectObservation = observation as? VNRecognizedObjectObservation else {
                        continue
                    }
                    let cropsize = VNImageRectForNormalizedRect(objectObservation.boundingBox, Int((image.size.width)), Int((image.size.height)))
                    let topLabelObservation = objectObservation.labels[0]
                    guard let cgImage = image.cgImage(forProposedRect: nil, context: nil, hints: nil) else{break}
                    guard let cutImageRef: CGImage = cgImage.cropping(to:cropsize)else {break}
                    let sie = NSSize(width: cropsize.width,height: cropsize.height)
                    let objectImg = NSImage(cgImage: cutImageRef, size: sie)
                    if objectImg.save(as: "CroppedImage\(picCount)") {
                        picCount += 1
                    }
                }
            }
        })
        objectRecognition.imageCropAndScaleOption = .scaleFill
        guard let cgImage = image.cgImage(forProposedRect: nil, context: nil, hints: nil) else{
            print("Failed to get cgimage from input image")
            return
        }
        let handler = VNImageRequestHandler(cgImage: cgImage, options: [:])
        do {
            try handler.perform([objectRecognition])
        } catch {
            print(error)
        }
        requests = [objectRecognition]
    } catch let error as NSError {
        print("Model loading went wrong: \(error)")
    }
}

Screenshot of Predictions tab

Upvotes: 3

Views: 323

Answers (1)

Matthijs Hollemans
Matthijs Hollemans

Reputation: 7892

You did not say what was wrong about the bounding boxes, but my guess is that they are correct but they're simply not being drawn in the right place. I wrote a blog post about this: https://machinethink.net/blog/bounding-boxes/

Upvotes: 2

Related Questions