CODEWITHSUNDEEP
swiftcoremlswift-package-manager
Vincent Garcia
Vincent Garcia

Reputation: 817

How to add a CoreML model into a Swift package?

I'm trying to write a Swift package that uses a CoreML model. I'm not very familiar with Swift packages creation and I could not make it work. Here is what I've done based on the different posts I've read so far:

  1. Create an empty package
$ mkdir MyPackage
$ cd MyPackage
$ swift package init
$ swift build
$ swift test

  1. Open the Package.swift file with XCode

  2. Drag and drop the MyModel.mlmodel file into the folder Sources/MyPackage

When I click on the MyModel.mlmodel file in XCode, I have the following message displayed under the class name:

Model is not part of any target. Add the model to a target to enable generation of the model class.

Similarly, if I use the command swift build in a Terminal I get the following message:

warning: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /Path/To/MyPackage/Sources/MyPackage/MyModel.mlmodel
  1. To solve this, I added MyModel into the target resources in the file Package.swift:
.target(
    name: "MyPackage",
    dependencies: [],
    resources: [.process("MyModel.mlmodel")]),

If I now use the command $ swift build, I don't have the warning anymore and I get the message:

[3/3] Merging module MyPackage

But when I check the MyModel.mlmodel file in XCode, I have the following message displayed under the class name:

Model class has not been generated yet.
  1. To solve this, I used the following command in a Terminal:
$ cd Sources/MyPackage 
$ xcrun coremlcompiler generate MyModel.mlmodel --language Swift .

This generated a MyModel.swift file next to the mlmodel file.

  1. I plugged the model in the code MyPackage.swift:
import CoreML

@available(iOS 12.0, *)
struct MyPackage {
    var model = try! MyModel(configuration: MLModelConfiguration())
}
  1. Finally, in the test file MyPackageTests.swift, I create an instance of MyPackage:
import XCTest
@testable import MyPackage

final class MyPackageTests: XCTestCase {
    func testExample() {
        if #available(iOS 12.0, *) {
            let foo = MyPackage()
        } else {
            // Fallback on earlier versions
        }
    }

    static var allTests = [
        ("testExample", testExample),
    ]
}

I get the following error (it seems that the CoreML model was not found):

Thread 1: Fatal error: Unexpectedly found nil while unwrapping an Optional value

I must have missed something... I hope my description was clear and detailed enough. Thank you for your help!

Upvotes: 6

Views: 5603

Answers (3)

sciasxp
sciasxp

Reputation: 1041

Instead of precompile your model you can compile on the fly:

if let url = Bundle.module.url(forResource: "MyModel", withExtension: "mlmodel") {
    let compiledURL = try! MLModel.compileModel(at: url)
    let model = try! MLModel(contentsOf: compiledURL, configuration: MLModelConfiguration())
} else if let url = Bundle.module.url(forResource: "MyModel", withExtension: "mlmodelc") {
    let model = try! MLModel(contentsOf: url, configuration: MLModelConfiguration())
}

You still need to add resources to your target:

targets: [
    .target(
        name: "MyPackage",
        dependencies: [],
        resources: [.process("MyMLFolder")]]
    ),
    .testTarget(
        name: "MyPackage Tests",
        dependencies: ["MyPackage"]
    ),
]

Upvotes: 3

Vincent Garcia
Vincent Garcia

Reputation: 817

The solution described bellow worked for me. I hope it is correct.

Conversion of the MLModel

The MLModel cannot be used directly in the Swift package. It first needs to be converted.

$ cd /path/to/folder/containg/mlmodel
$ xcrun coremlcompiler compile MyModel.mlmodel .
$ xcrun coremlcompiler generate MyModel.mlmodel . --language Swift

The first xcrun command will compile the model and create a folder named MyModel.mlmodelc. The second xcrun command will generate a MyModel.swift file.

Add the model to the Swift package

We consider that a Swift package already exists and is located in /path/to/MyPackage/.

  1. Copy the MyModel.mlmodelc folder and MyModel.swift file into the folder /path/to/MyPackage/Sources/MyPackage
  2. Add the MyModel.mlmodelc in the target resources in the file Package.swift:
.target(
    name: "MyPackage",
    dependencies: [],
        resources: [.process("MyModel.mlmodelc")]),

Instantiate MyModel

In the Swift code, simply create an instance of MyModel:

let model = try? MyModel(configuration: MLModelConfiguration())

or:

let url = Bundle.module.url(forResource: "MyModel", withExtension: "mlmodelc")!
let model = try? MyModel(contentsOf: url, configuration: MLModelConfiguration())

Troubleshooting

I got a Type 'MLModel' has no member '__loadContents' error at first. This seems to be a bug related to XCode 12. I simply commented the 2 functions that caused a problem.

See here and here for more information.

Upvotes: 9

Matthijs Hollemans
Matthijs Hollemans

Reputation: 7892

This line is probably where it goes wrong: var model = try! MyModel(configuration: MLModelConfiguration())

Since you've added a mlmodel file to the package, it hasn't been compiled yet. I'm not an expert on Swift packages, but I don't believe Xcode automatically compiles this model now. You can see this for yourself when you open up the compiled app bundle -- does it have the mlmodel file in it or the mlmodelc folder?

You may need to add the mlmodelc to the package, not the mlmodel. You can create this by doing:

$ xcrun coremlcompiler compile MyModel.mlmodel .

Next, in your app you will need to load the model as follows:

let url = YourBundle.url(forResource: "MyModel", withExtension: "mlmodelc")!
let model = try! MyModel(contentsOf: url, configuration: MLModelConfiguration())

where YourBundle is a reference to the bundle that contains the mlmodelc file (which I guess is the bundle for the Swift package).

Upvotes: 2

Related Questions