CODEWITHSUNDEEP
pythontensorflowkerasgoogle-cloud-ml
Ciprian Andrei Focsaneanu
Ciprian Andrei Focsaneanu

Reputation: 559

Reduce size of Tensorflow SavedModel for Google ML Engine deployment

I have developed and trained a CNN Keras model and now I want to deploy this model to Google Machine Learning Engine, so I can execute predictions using their API.

I have converted to SavedModel format and the export/saved_model.pb has 14MB and the /export/variables/ directory has around 380MB. Google ML Engine has a limit of 250MB for this data and does not allow deploying a bigger model.

I saw a solution regarding https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms, but I still didn't manage to bezel build this project due to VS unmet dependencies.

Is there any other way to reduce/compress (especially) the variables directory? What I would want is to convert dtype from int64 to int32, but don't know the format of variables.data-00000-of-00001 file.

Thanks a lot!

I attach my Keras model to Tensorflow SavedModel code here:

# reset session
K.clear_session()
sess = tf.Session()
K.set_session(sess)

# disable loading of learning nodes
K.set_learning_phase(0)

# load model
model = load_model('local-activity-recognition-model.h5')
config = model.get_config()
weights = model.get_weights()
new_Model = Model.from_config(config)
new_Model.set_weights(weights)

# export saved model
export_path = '.' + '/export'
builder = saved_model_builder.SavedModelBuilder(export_path)

signature = predict_signature_def(inputs={'export_input': new_Model.input},
                              outputs={'export_output': new_Model.output})

with K.get_session() as sess:
    builder.add_meta_graph_and_variables(sess=sess,
                                     tags=[tag_constants.SERVING],
                                     signature_def_map={
                                                 signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature})
builder.save()

Upvotes: 1

Views: 1073

Answers (2)

T. Kelher
T. Kelher

Reputation: 1186

You can freeze the graph, that should shrink it down a bit. https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph_test.py

If you're building a classifier, you might want to step to InceptionV3 architecture, this can easily be trained by tensorflows retrain code. This architecture is only 90mb. https://www.tensorflow.org/tutorials/image_retraining https://github.com/tensorflow/hub/blob/master/examples/image_retraining/retrain.py https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/label_image/label_image.py

Upvotes: 1

rhaertel80
rhaertel80

Reputation: 8399

Is it possible in your model to set the dtype of the Variables when building the graph? Using float32 while training is generally a good idea.

You can also use the techniques described here, but they take a little more effort.

Upvotes: 0

Related Questions