CODEWITHSUNDEEP
debuggingsavetensorflow
thang
thang

Reputation: 3466

Is there a way to save an intermediate output in Tensorflow to a file?

I have a conv net that does a bunch of things.

The inference code looks something like this:

conv0_feature_count = 2
with tf.variable_scope('conv0') as scope:
    kernel = _variable_with_weight_decay('weights', shape=[5, 5, 1, conv0_feature_count], stddev=5e-2, wd=0.0)
    conv = tf.nn.conv2d(images, kernel, [1, 1, 1, 1], padding='SAME')
    biases = _variable_on_cpu('biases', [conv0_feature_count], tf.constant_initializer(0.0))
    bias = tf.nn.bias_add(conv, biases)
    conv0 = tf.nn.relu(bias, name=scope.name)

And so on and so forth followed by max pooling, etc.

I would like to be able to save conv0 to a file for inspection. I know that tf.Print would print the values in the console, but it is a big tensor. Printing the value makes no sense.

Upvotes: 3

Views: 3820

Answers (1)

mrry
mrry

Reputation: 126154

TensorFlow doesn't have any public operators that make it possible to write a tensor to a file, but if you look at the implementation of tf.train.Saver (in particular the BaseSaverBuilder.save_op() method), you will see that it includes an implementation of an op that can write one or more tensors to a file, and it is used to write checkpoints.

CAVEAT: The following solution relies on an internal implementation and is subject to change, but works with TensorFlow r0.10rc0.

The following snippet of code writes the tensor t to a file called "/tmp/t.ckpt":

import tensorflow as tf
from tensorflow.python.ops import io_ops

t = tf.matmul(tf.constant([[6, 6]]), tf.constant([[2], [-1]]))

save_op = io_ops._save(filename="/tmp/t.ckpt", tensor_names=["t"],
                       tensors=[t])

sess = tf.Session()
sess.run(save_op)  # Writes `t` to "/tmp/t.ckpt".

To read the value you can use tf.train.NewCheckpointReader() as follows:

reader = tf.train.NewCheckpointReader("/tmp/t.ckpt")
print reader.get_tensor("t")  # ==> "array([[6]], dtype=int32)"

Upvotes: 5

Related Questions