CODEWITHSUNDEEP
pythontensorflowtensorflow-litemxnetgluon
kmf
kmf

Reputation: 117

Using GluonCV models in TensorFlow (Lite)

I'm working on the deployment of models on embedded devices, making performance comparisons and the like. This is an internship, so I'm really constrained on time and can't go about re-implementing / re-trained models but I have to use what is available (I actually asked this explicitly to my supervisor). Since TorchScript is not really as mature as TF Lite, at least from what I've gathered, I'm going with the latter. It's going well with pre-2018 models, but many SotA models like ResNeSt only have code in PyTorch. GluonCV, however, seems to provide a nice selection of models in their zoo and is based on TensorFlow, so I thought there'd be a way of exporting those to a SavedModel, a Keras .h5 or whatever, but I found none after a lot of searching. I found MMdnn but trying it on JSON exported models fails during conversion to IR (I'm attaching the output at the bottom, it seems that MXNet JSON and Gluon JSON are not the same format).

Has anybody else worked with exporting Gluon models to the wild? How did it go?

Thank you!

Output of mmtoir -f mxnet -n resnest200-symbol.json -d resnest200 --inputShape 3,257,257:

/home/kmfrick/Gluon_Tinkering/venv/lib/python3.8/site-packages/mxnet/module/base_module.py:55: UserWarning: You created Module with Module(..., label_names=['softmax_label']) but input with name 'softmax_label' is not found in symbol.list_arguments(). Did you mean one of:
    data
    _defaultpreprocess1_init_mean
    _defaultpreprocess1_init_scale
  warnings.warn(msg)
Warning: MXNet Parser has not supported operator null with name data.
Warning: convert the null operator with name [data] into input layer.
Warning: MXNet Parser has not supported operator null with name _defaultpreprocess1_init_scale.
Warning: convert the null operator with name [_defaultpreprocess1_init_scale] into input layer.
terminate called after throwing an instance of 'dmlc::Error'
  what():  [09:24:49] src/c_api/c_api_symbolic.cc:540: InferShapeKeyword argument name data not found.
Candidate arguments:
    [0]_defaultpreprocess1_init_scale

Stack trace:
  [bt] (0) /home/kmfrick/Gluon_Tinkering/venv/lib/python3.8/site-packages/mxnet/libmxnet.so(+0x307d3b) [0x7f0127eb9d3b]
  [bt] (1) /home/kmfrick/Gluon_Tinkering/venv/lib/python3.8/site-packages/mxnet/libmxnet.so(+0x33a3755) [0x7f012af55755]

Upvotes: 1

Views: 510

Answers (1)

Olivier Cruchant
Olivier Cruchant

Reputation: 4037

Gluoncv is an excellent MXNet-based toolkit for computer vision! Several options to deploy gluoncv models to embedded runtimes:

  1. You can use ONNX to convert models to other runtimes, for example CoreML for iOS or NNAPI for Android
  2. You can use TVM
  3. You can use SageMaker Neo + DLR runtime, probably the easiest solution. The git includes examples for Android.

Keep in mind that compilation and portability from a framework to another depends on operators coverage, it may not work for exotic or very recent models

Upvotes: 1

Related Questions