CODEWITHSUNDEEP
kerascoremlcoremltools
lreichold
lreichold

Reputation: 765

Keras to Core ML model conversion with custom layer - conversion function ignored

I am attempting to convert a model that's been trained via Keras with TensorFlow 1.x as the backend into Core ML format (.mlmodel). I have the full source and weights for the model, as well as a frozen Keras graph (.h5 file). My model (see architecture here) is defined using the Keras functional API, and has one custom layer called AttentionWeightedAverage.

When running the following conversion code, I receive a ValueError: Unknown layer: AttentionWeightedAverage.

import coremltools
mlmodel = coremltools.converters.keras.convert('deepmoji_model.h5')

Naturally, since this is a custom layer (which also happens to have a single boolean hyperparameter called return_attention), I understood that I needed to tell Core ML how to handle it, so I implemented the following, based on Matthijs Hollemans' great blog post:

class AttentionWeightedAverage(Layer):
    # class defined inline here ...
    # https://github.com/bfelbo/DeepMoji/blob/master/deepmoji/attlayer.py

def convert_ATTN(layer):
    params = NeuralNetwork_pb2.CustomLayerParams()
    params.className = "AttentionWeightedAverage"
    params.description = "A fancy new activation function"

    params.parameters["return_attention"].boolValue = layer.return_attention
    return params


mlmodel = coremltools.converters.keras.convert('deepmoji_model.h5',
                                               add_custom_layers=True,
                                               custom_conversion_functions={"AttentionWeightedAverage": convert_ATTN}
                                              )

However, I am still receiving the same "Unknown Layer" error as above when attempting to run the conversion. What could be causing the conversion script not to recognize the conversion function I've provided?

I'm running coremltools == 3.3 (latest) with keras == 2.3.1. Any guidance is much appreciated!

Upvotes: 0

Views: 958

Answers (1)

lreichold
lreichold

Reputation: 765

It turns out, the Unknown layer error was originating from Keras itself, which was unable to successfully load_model since it couldn't deserialize the custom layer. If instead we pass a fully deserialized model (instead of just the file path) to the converter, the converter runs without issue.

model = load_model('deepmoji_model.h5', custom_objects={'AttentionWeightedAverage': AttentionWeightedAverage()})

mlmodel = coremltools.converters.keras.convert(model,
                                               add_custom_layers=True,
                                               custom_conversion_functions={"AttentionWeightedAverage": convert_ATTN}
                                              )
mlmodel.save('deepmoji_model.mlmodel')

Upvotes: 2

Related Questions