CODEWITHSUNDEEP
iosswiftapple-vision
Tamim Dari Chowdhury
Tamim Dari Chowdhury

Reputation: 111

Is "VNFeaturePrintObservation" and vision distance computation different for two images in two different iOS versions like iOS 16.0 and iOS 17.0?

I'm trying to determine how similar two images are. For that, I'm using VNGenerateImageFeaturePrintRequest and generating a VNFeaturePrintObservation from an image. Then for the two different images, I'm computing feature distance by using their VNFeaturePrintObservation. Unfortunately, computed distances are very different for iOS 16.0 and iOS 17.0 thus failing my threshold value.

I tried to get featurePrintObservation for images like below,

let requestHandler = VNImageRequestHandler(cgImage: cgImage, options: [:])       
    let request = VNGenerateImageFeaturePrintRequest()
       do {               
          try requestHandler.perform([request])              
          self.feature = request.results?.first
        } catch {
            print("Vision error: \(error)")
       }

Then I tried to compute distance between two feature like below,

try! feature1.computeDistance(&visionDistance, to: feature2)

The output result visionDistance is different for iOS 16.0 and iOS 17.0 for the same two images (For example, 22.66224 in iOS 16.3.1 and 1.2178229 in iOS 17.0.1). Has VNFeaturePrintObservation has changed in iOS 17.0, and if so then is there any way to generalize the calculation for both versions of iOS?

Upvotes: 8

Views: 804

Answers (1)

evergreen_herb
evergreen_herb

Reputation: 111

A colleague and I wrote a detailed Medium article about VNGenerateImageFeaturePrintRequest and the differences between iOS 16 and iOS 17

To sum up the differences:

  • On iOS 16, a VNFeaturePrintObservation is a non-normalized vector containing 2048 Float.
  • On iOS 17, it's a normalized vector having length 768.

Also, the computeDistance(_:to:) method computes the straightforward Euclidean distance between those vectors.

As a consequence:

  • On iOS 17, the distance between two observations is always less than 2.0.
  • On iOS 16, the distance can vary a lot; typical values range between 0.0 and 40.0.

So, yes, this change can be quite annoying. Note that you can reproduce the iOS 16 behavior on iOS 17 by setting the revision property of a VNGenerateImageFeaturePrintRequest instance to VNGenerateImageFeaturePrintRequestRevision1 (distance values will not be exactly the same though, you'll get slight, non-significant differences).

If you have to support only iOS 17, I would recommend to prefer the iOS 17 VNGenerateImageFeaturePrintRequestRevision2 algorithm, since comparing normalized vectors makes much more sense (the distance between normalized vectors is closely related to their cosine distance, a popular measure of vector similarity in data analysis).

Upvotes: 11

Related Questions