Reputation: 673
I'm working with hyperspectral image of size [610, 340, 102] from here saved in .mat file. After loading it with scipy, it is represented as array. Prior to make computations I have to reshape it to [610 * 340, 102]. The code below is running on desktop machine with i5-6400 processor:
import tensorflow as tf
import scipy.io as spio
import time
pavia_u = spio.loadmat("./../data/PaviaU.mat")["paviaU"]
original_shape = tf.shape(pavia_u)
pavia_u_reshaped = tf.to_float(tf.squeeze(tf.reshape(pavia_u, [-1, original_shape[0] * original_shape[1], original_shape[2]])))
with tf.Session() as session:
start = time.clock()
pavia_u_reshaped_value = session.run(pavia_u_reshaped)
end = time.clock()
print("Reshape time: {}".format(end - start))
and it prints: Reshape time: 157.7449091748693
It seems that reshape operation on so relative small dataset should not be so slow. Anyway, I moved reshaping to numpy a
pavia_u = spio.loadmat("./../data/PaviaU.mat")["paviaU"]
original_shape = pavia_u.shape
pavia_u_reshaped = pavia_u.reshape([-1, original_shape[0] * original_shape[1], original_shape[2]]).squeeze())
with tf.Session() as session:
start = time.clock()
session.run(tf.identity(pavia_u_reshaped))
end = time.clock()
print("Test time: {}".format(end - start))
which prints: Test time: 143.26529380869727
What consumes so much time now? How to fix this.
Upvotes: 0
Views: 515
Reputation: 17201
Reshape is not the reason for the slow time, its the initialisations that is taking time. Changing the code to :
original_shape = tf.shape(pavia_u)
X = tf.Variable(pavia_u, dtype=tf.float32)
pavia_u_reshaped = tf.to_float(tf.squeeze(tf.reshape(X, [-1, original_shape[0] * original_shape[1], original_shape[2]])))
sess = tf.InteractiveSession()
start = time.clock()
tf.global_variables_initializer().run()
end = time.clock()
print("Initialization time: {}".format(end - start))
start = time.clock()
pavia_u_reshaped_value = sess.run(pavia_u_reshaped)
end = time.clock()
print("Reshape time: {}".format(end - start))
# Initialization time: 734.495346
# Reshape time: 1.897106
Upvotes: 1