Error on loading keras model with deeplearning4j

Question

I have been struggling with loading my keras neural network model for my Android application with deeplearning4j for a while now. I have searched for solutions (as much as there are), but every solution brings up new errors and I just could not get this thing to work.

Anyways, I have trained a NON sequential model with keras in Python and saved it like this:

model.save('model.h5')

Now I am trying to import this model with deeplearning4j in Android Studio. I have tried many possible variants, but this is where I am now:

String modelPath = new ClassPathResource("res/raw/model.h5").getFile().getPath();
ComputationGraph model = KerasModelImport.importKerasModelAndWeights(modelPath)

This however triggers a following error:

java.lang.NoClassDefFoundError: Failed resolution of: Lorg/bytedeco/javacpp/hdf5;

As I understand, gradle is unable to resolve a dependency hdf5 from org.bytedeco, which I agree on as I have excluded hdf5-platform in my gradle build, but hdf5 should not even be supported by Android as far as I know (?).

I have also tried to include hdf5-platform and run the same code, but doing so triggers another error:

java.lang.UnsatisfiedLinkError: Platform "android-arm64" not supported by class org.bytedeco.javacpp.hdf5

I am rather new into gradle concepts and I do not know Android in depth, but seems that the problem is with my gradle dependencies. There is also limited amount of information about deeplearning4j and I could not figure out an alternative solution either.

I will also include my gradle dependencies I have from this tutorial.

implementation (group: 'org.deeplearning4j', name: 'deeplearning4j-core', version: '1.0.0-beta3') {
    exclude group: 'org.bytedeco.javacpp-presets', module: 'opencv-platform'
    exclude group: 'org.bytedeco.javacpp-presets', module: 'leptonica-platform'
    exclude group: 'org.bytedeco.javacpp-presets', module: 'hdf5-platform'
    exclude group: 'org.nd4j', module: 'nd4j-base64'
}
implementation group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta3'
implementation group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta3', classifier: "android-arm"
implementation group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta3', classifier: "android-arm64"
implementation group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta3', classifier: "android-x86"
implementation group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta3', classifier: "android-x86_64"
implementation group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3'
implementation group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-arm"
implementation group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-arm64"
implementation group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-x86"
implementation group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-x86_64"
implementation group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.3-1.4.3'
implementation group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.3-1.4.3', classifier: "android-arm"
implementation group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.3-1.4.3', classifier: "android-arm64"
implementation group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.3-1.4.3', classifier: "android-x86"
implementation group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.3-1.4.3', classifier: "android-x86_64"
implementation group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.76.0-1.4.3'
implementation group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.76.0-1.4.3', classifier: "android-arm"
implementation group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.76.0-1.4.3', classifier: "android-arm64"
implementation group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.76.0-1.4.3', classifier: "android-x86"
implementation group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.76.0-1.4.3', classifier: "android-x86_64"

(How) should I change my dependencies to get this model importing to work?

Or should I change the way of importing my model somehow?

Answer 1

You can try to load the model with TF Lite. To load a TensorFlow Keras model in Android or even iOS, you can use TensorFlow Lite.

First, you need to convert your Keras ( .h5 ) model to a TFLite model ( .tflite )

import tensorflow as tf

converter = tf.lite.TFLiteConverter.from_keras_model_file( 'model.h5' )
tflite_model = converter.convert()
open( 'model.tflite' , 'wb' ).write( tflite_model )

You may do the following :

1. If your model needs to be hosted on a cloud source which will be downloaded by your app then you can use Firebase ML Kit. For custom TFLite models read here.

2. You can keep the TFLite model in the app's assets folder and then load the MappedByteBuffer of it. The TensorFlow Lite dependency for Android is available :

   implementation ‘org.tensorflow:tensorflow-lite:2.3.0’
   

You can refer to this codelab and this article.

You can load the MappedByteBuffer like :

private MappedByteBuffer loadModelFile(Activity activity) throws IOException {
  AssetFileDescriptor fileDescriptor = activity.getAssets().openFd(getModelPath());
  FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
  FileChannel fileChannel = inputStream.getChannel();
  long startOffset = fileDescriptor.getStartOffset();
  long declaredLength = fileDescriptor.getDeclaredLength();
  return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
}