CODEWITHSUNDEEP
iosswiftmemory-leakstext-recognitionapple-vision
Dave
Dave

Reputation: 91

Working with Vision text recognition (Swift) on IOS 15.6 and memory grows ~25MB per recognition. Anyway to free up memory?

I need to do text recognition on hundreds of images one at a time, but each time, memory grows by ~25mb+. I've searched the internet but can't find what is causing the memory retention or how to release it. By putting in breaks, I can see that the jump in size occurs with each call to imageRequestHandler.perform(). Below is the relevant code. Any suggestions?

    func textRecognition(image:CGImage) {
                
        let textRecognitionRequest = VNRecognizeTextRequest(
            completionHandler: self.handleDetectedText)
        textRecognitionRequest.recognitionLevel = .accurate
        textRecognitionRequest.recognitionLanguages = ["en_US"]
        textRecognitionRequest.usesLanguageCorrection = false
        
        // request handler
        let textRequest = [textRecognitionRequest]
        let imageRequestHandler = VNImageRequestHandler(cgImage: image, orientation: .up, options: [:])
        DispatchQueue.global(qos: .userInitiated).async {
            do {
                // perform request
                try imageRequestHandler.perform(textRequest)
            } catch let error {
                print("Error \(error)")
            }
        }
    }
    
    func handleDetectedText(request: VNRequest?, error:Error?){
        if let error = error {  print("ERROR: \(error)"); return  }
        guard let results = request?.results, results.count > 0 else {
            DispatchQueue.main.async {
                self.result_field.isEnabled=false
                self.result_field.text = "Scan failed - Retry"
                let desc = NSMutableAttributedString(string: "Retake Photo", attributes: [NSAttributedString.Key.font: UIFont.systemFont(ofSize: 22, weight: .regular)])
                self.take_button.setAttributedTitle(desc, for: UIControl.State.normal)
                self.take_button.isHidden = false
                self.take_button.isEnabled = true
            }
            return // code to process the text replaced by the 'return' statement
        }}}

Upvotes: 0

Views: 828

Answers (1)

timbre timbre
timbre timbre

Reputation: 13995

I would recommend profiling to see what takes the memory. However in general I see 2 areas that could be explored for potential improvements:

  1. Image may be causing higher memory usage. Our (developers) tendency is to capture "the best" image. However text recognition may not need a huge high quality image (and give the same results for smaller image). So try to make your image smaller on capturing and/or processing:

    • experiment with AVCapturePhotoSettings, namely photoQualityPrioritization and photo format,
    • try to reduce CGImage size / maybe make it B&W. Those operations are not without the cost as well though. Or try to pass CVPixelBuffer directly to VNImageRequestHandler (without converting to CGImage - that's if you are taking image from camera)

  2. See if autoreleasepool benefits your memory usage. The most obvious location is imageRequestHandler.perform: try to perform it on a dedicated queue, and set autoreleaseFrequency: .workItem for that queue:

private static let performQueue = DispatchQueue(
        label: "your label",
        qos: .userInitiated,
        autoreleaseFrequency: .workItem, // <-- switching from default .never
        target: .global(qos: .userInitiated) <-- same priority as the one you have
    )

// ...

Self.performQueue.async {
    try imageRequestHandler.perform(textRequest)
}

What does it do:

The queue configures an autorelease pool before the execution of a block, and releases the objects in that pool after the block finishes executing.

Since the recognition request may have a lot of temporary objects, you may benefit from this setting, since the objects will be immediately released after the request completes (instead of "eventually"). There's no guarantee that it will help, but worth a try.

Again, those are suggestions, but really performance evaluation is needed if reducing memory is that important for you. Although I think 25MB is quite reasonable, to be honest.

Upvotes: 1

Related Questions