CODEWITHSUNDEEP
tensorflow
n00b
n00b

Reputation: 1639

Optimize Tensorflow for multiple runs

I need to execute the statement, sess.run() multiple times. I create the sess once at the beginning of my code. However, each sess.run() statement takes almost 0.5-0.8 seconds on my CPU machine. Is there any way I can optimize this? Since Tensorflow does lazy loading, is there any way I can make it not do it, and make this faster?

I'm using the Inception model from the image classifying example.

def load_network():
    with gfile.FastGFile('model.pb', 'rb') as f:
        graph_def = tf.GraphDef()
        data = f.read()
        graph_def.ParseFromString(data)
        png_data = tf.placeholder(tf.string, shape=[])
        decoded_png = tf.image.decode_png(png_data, channels=3)
        _ = tf.import_graph_def(graph_def, name=input_map={'DecodeJpeg': decoded_png})
        return png_data

def get_pool3(sess, png_data, imgBuffer):
    pool3 = sess.graph.get_tensor_by_name('pool_3:0')
    pool3Vector = sess.run(pool3, {png_data: imgBuffer.getvalue()})
    return pool3Vector

def main():
    sess = getTensorSession()
    png_data = load_network()

    # The below line needs to be called multiple times, which is what takes
    # nearly 0.5-0.8 seconds.
    # imgBuffer contains the stored value of the image.
    pool3 = get_pool3(sess, png_data, imgBuffer)

Upvotes: 1

Views: 849

Answers (1)

Peter Hawkins
Peter Hawkins

Reputation: 3211

Tensorflow runs operations lazily --- nothing is actually computed until sess.run() is called. When you call sess.run(), Tensorflow executes all of the operations in your computation graph. So if sess.run() is taking 0.5-0.8 seconds, it is likely that your computation itself is taking 0.5-0.8s.

(There is some overhead to sess.run(), but it shouldn't be anywhere near the order of half a second.)

Hope that helps!

Added:

Here are some things you might look into to speed your computation up:

  • use Tensorflow's profiling tools to look at what part of your computation is taking the time. They are not documented yet, but you can find some information about them in this github issue: https://github.com/tensorflow/tensorflow/issues/1824

  • make your computation cheaper --- reduce the complexity of your model, use smaller images, etc.

  • run your computation on a GPU instead of CPU.

Upvotes: 2

Related Questions