CODEWITHSUNDEEP
pythontensorflowtensorflow.jstensorflowjs-converter
Marie
Marie

Reputation: 146

How to load MobileNetV3 or EfficientNetB0 into a React app with TensorFlow.js while overcoming compatibility issues?

I have train two pre-trained models using Python : MobileNetV3Large and EfficientNetB0.

Here are my model :

base_model = tf.keras.applications.EfficientNetB0(weights="imagenet", include_top=False, input_shape=(image_size,image_size, 3))

inputs = tf.keras.Input(shape=(image_size, image_size, 3))

x = inputs

# Apply the base model to the input
x = base_model(x, training=False)

x = tf.keras.layers.GlobalAveragePooling2D()(x)
x = tf.keras.layers.Dense(128, activation="relu")(x)
x = tf.keras.layers.Dropout(0.5)(x)

# Add our own output layer
outputs = tf.keras.layers.Dense(4, activation="softmax")(x)

# Create a new model using the base model and the custom layers
model = tf.keras.Model(inputs=inputs, outputs=outputs)

The results are awesome ! I get more than 80-90% accuracy to classify images.

I was struggling the load them into my React App using Tensorflow JS:

import * as tf from '@tensorflow/tfjs';

export async function loadModel() {
  const model = await tf.loadLayersModel('/model_js/model.json');
  console.log(model);
  console.log('Model loaded successfully.');

  return model;
}

I had to convert them using the conversion wizard after they was saved using Saved Model. Impossible to load them correctly. I have having all sorts of error. Like about normalization layers or lambda layers that aren't compatible with TensorflowJS or

Uncaught (in promise) Error: Unknown layer: TFOpLambda. This may be due to one of the following reasons:
1. The layer is defined in Python, in which case it needs to be ported to TensorFlow.js or your JavaScript code.
2. The custom layer is defined in JavaScript, but is not registered properly with tf.serialization.registerClass().

I finally found out that only the MobileNetV2 was compatible for tensorflowjs.. But it gives me terrible result (no more than 40% accuracy on a 4 classes classifier).

I don't understand why I cannot find much data on training pre-trained model for tensorflow js. It seems that most of the documentation online is about using pre-trained model directly, without retraining them. I cannot find a list of 100% compatible pre-trained model for tensorflowjs that can be easily retrain by me.

Why MobileNetV3 isn't compatible ? Isn't a way to load it on tensorflowjs ?

Upvotes: 0

Views: 792

Answers (1)

Skyler
Skyler

Reputation: 1189

I was able to get a successful MobileNetV3 classification in react using this guide https://medium.com/@tesheng_93/image-classification-in-the-browser-with-tensorflow-js-react-and-github-pages-753cd9561543

In particular, I think there were 2 necessary changes to get passed the TFOpLambda issue you are running into:

  1. Convert the model to a graph type (note the output_format):
tensorflowjs_converter `
--input_format=tf_saved_model `
--output_format=tfjs_graph_model `
./path/to/your/model/folder `
./path/to/output/folder
  1. Use tfjs-converter's loadGraphModel()
import { loadGraphModel } from '@tensorflow/tfjs-converter';

async function loadModel() {
  const model = await loadGraphModel("./tfjs_model/model.json")
  return model;
}
  1. Small example of how to get predictions and predicted class idx
const infer = async (img) => {
  const img_size = 34
  const model = await loadModel();
  const normalized_img = tf.browser.fromPixels(img).resizeNearestNeighbor([img_size,img_size]).toFloat().expandDims();
  const results = model.predict(normalized_img);
  const predictions = results.arraySync();

  const classIdx = results.as1D().argMax().dataSync()[0];

  console.log(classIdx, predictions);
}

Note I'm still learning this myself so i'd suggest checking the article for specifics, for example it does some extra clean up steps for the prediction which I haven't looked into yet.

Upvotes: 0

Related Questions