CODEWITHSUNDEEP
pythoniosswifttensorflowcoreml
Kautham Krishna
Kautham Krishna

Reputation: 967

How to convert Tensorflow Simple Audio Recognition frozen graph(.pb) to Core ML model?

I have been trying to implement the Tensorflow's simple audio recognition to an iphone app. After some research i found that i need to convert the Tensorflow's frozen graph .pb file to core ML model and then use it in the iOS apps. So i tried following this sample and referenced this convertor. But it looks like the convertor was mainly written to convert models that take image as input.But My model should be able to take audio .wav files as input. `

import tfcoreml as tf_converter
tf_converter.convert(tf_model_path = 'my_frozen_graph.pb',
                     mlmodel_path = 'my_model.mlmodel',
                     output_feature_names = ['labels_softmax:0'],
                     class_labels = 'classes.txt'
                     )

When i try to convert my graph to core ML model using the above code i get the following error response. 

(env3) minimaci73$ python model.py
WARNING:root:Keras version 2.2.0 detected. Last version known to be fully compatible of Keras is 2.1.6 .
WARNING:root:TensorFlow version 1.8.0 detected. Last version known to be fully compatible is 1.5.0 .

Loading the TF graph...
Graph Loaded.
Traceback (most recent call last):
  File "/Users/minimaci73/anaconda3/envs/env3/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py", line 204, in _convert_pb_to_mlmodel
    shape_list = shape.as_list()
  File "/Users/minimaci73/anaconda3/envs/env3/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 900, in as_list
    raise ValueError("as_list() is not defined on an unknown TensorShape.")
ValueError: as_list() is not defined on an unknown TensorShape.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "model.py", line 6, in <module>
    class_labels = 'conv_labels.txt'
  File "/Users/minimaci73/anaconda3/envs/env3/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py", line 586, in convert
    custom_conversion_functions=custom_conversion_functions)
  File "/Users/minimaci73/anaconda3/envs/env3/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py", line 206, in _convert_pb_to_mlmodel
    raise ValueError('Please provide the shape for the input {} through the argument \'input_name_shape_dict\''.format(input_name))
ValueError: Please provide the shape for the input wav_data:0 through the argument 'input_name_shape_dict'
(env3) minimaci73$

As per the last error message I need to enter the input shape of the input wav_data:0. So i created the text summary of the frozen graph as mentioned in the tutorial and looked up on the wav_data input shape. But the input shape is <unknown> for this.

enter image description here

I would like to know if there are any other methods to convert this file into core ML model? or Is there any way to use the frozen graph .pb file directly in an ios swift app ?

The Tensorflow-experimental Swift framework is also poorly documented. If you guys have any good resource on this, please share.

Update As per the console log i downgraded the keras and tensorflow versions and also as per Matthijs answer i have added the input_name_shape_dict = { "import/wav_data:0" : [1, 16, 44100, 1] Now the console shows the error in different way 

(env3) minimaci73$ python model.py

Loading the TF graph...
Traceback (most recent call last):
  File "model.py", line 7, in <module>
    class_labels = 'conv_labels.txt'
  File "/Users/minimaci73/anaconda3/envs/env3/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py", line 586, in convert
    custom_conversion_functions=custom_conversion_functions)
  File "/Users/minimaci73/anaconda3/envs/env3/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py", line 153, in _convert_pb_to_mlmodel
    tf.import_graph_def(gdef, name='')
  File "/Users/minimaci73/anaconda3/envs/env3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 316, in new_func
    return func(*args, **kwargs)
  File "/Users/minimaci73/anaconda3/envs/env3/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 541, in import_graph_def
    raise ValueError('No op named %s in defined operations.' % node.op)
ValueError: No op named DecodeWav in defined operations.
(env3) minimaci73$ python model.py

Upvotes: 4

Views: 815

Answers (1)

Matthijs Hollemans
Matthijs Hollemans

Reputation: 7892

You can supply an input shape to the converter:

input_name_shape_dict={ "import/wav_data:0" : [1, input_height, input_width, channels] })

If your data is 1-dimensional, then the height and width should be 1 and channels should be the length of the data.

You may run into other conversion errors too, but this is the first step. :-) (tfcoreml needs to know the input shape so that it can run the model with some fake data.)

Upvotes: 2

Related Questions