CODEWITHSUNDEEP
pythonpython-3.xtensorflow
Shubhang Desai
Shubhang Desai

Reputation: 319

Confusion about TensorFlow shape rank

I know there are already questions with similar titles, but before you report this as a repeat, please allow me to say that all of the answer to those questions are extremely ad hoc and not applicable to my problem.

I'm having trouble understanding why I cannot take the matrix multiplication (well, technically the matrix-vector multiplication) of two Tensors in TensorFlow. I have a Tensor v with shape (1000, 1000) and another Tensor h_previous with shape (1000). I'm doing plenty of matrix multiplications with two Tensors of the exactly same shapes in the program before, but this is just throwing a cryptic error. Here are the critical parts of the graph:

# Variables
# Encoder input
X = tf.placeholder(tf.float32, shape=[k, None])
we = tf.Variable(tf.truncated_normal([500, k], -0.1, 0.1))
# Encoder update gate
wz = tf.Variable(tf.truncated_normal([1000, 500], -0.1, 0.1))
uz = tf.Variable(tf.truncated_normal([1000, 1000], -0.1, 0.1))
# Encoder reset gate
wr = tf.Variable(tf.truncated_normal([1000, 500], -0.1, 0.1))
ur = tf.Variable(tf.truncated_normal([1000, 1000], -0.1, 0.1))
# Encoder h~ [find name]
w = tf.Variable(tf.truncated_normal([1000, 500], -0.1, 0.1))
u = tf.Variable(tf.truncated_normal([1000, 1000], -0.1, 0.1))
# Encoder representation weight
v = tf.Variable(tf.truncated_normal([1000, 1000], -0.1, 0.1))

# Encoder
h_previous = tf.zeros([1000])
for t in range(N):
    # Current vector and its embedding
    xt = tf.reshape(tf.slice(X, [t, 0], [1, k]), [k])
    e = tf.matmul(we, xt)
    # Reset calculation
    r = tf.sigmoid(tf.matmul(wr, e) + tf.matmul(ur, h_previous))
    # Update calculation
    z = tf.sigmoid(tf.matmul(wz, e) + tf.matmul(uz, h_previous))
    # Hidden-tilde calculation
    h_tilde = tf.tanh(tf.matmul(w, e) + tf.matmul(u, r * h_previous))
    # Hidden calculation
    one = tf.ones([1000])
    h = z * h_previous + (one - z) * h_tilde
    h_previous = h
c = tf.tanh(tf.matmul(v, h_previous))

I'm stumped. Does anyone have any clue? Thanks in advance. :)

Upvotes: 0

Views: 148

Answers (1)

Gregory Begelman
Gregory Begelman

Reputation: 554

I have fixed your code in a couple of places and now it works (see below). Generally, the inputs for tf.matmul should be two 2D matrices (see docs here), while you passed a 2-D matrix (size is 1000x1000) and a 1-D matrix (size is 1000). If you reshape the second matrix to 1000x1 or 1x1000, matmul will work.

k = 77
N = 17
# Variables
# Encoder input
X = tf.placeholder(tf.float32, shape=[k, None])
we = tf.Variable(tf.truncated_normal([500, k], -0.1, 0.1))
# Encoder update gate
wz = tf.Variable(tf.truncated_normal([1000, 500], -0.1, 0.1))
uz = tf.Variable(tf.truncated_normal([1000, 1000], -0.1, 0.1))
# Encoder reset gate
wr = tf.Variable(tf.truncated_normal([1000, 500], -0.1, 0.1))
ur = tf.Variable(tf.truncated_normal([1000, 1000], -0.1, 0.1))
# Encoder h~ [find name]
w = tf.Variable(tf.truncated_normal([1000, 500], -0.1, 0.1))
u = tf.Variable(tf.truncated_normal([1000, 1000], -0.1, 0.1))
# Encoder representation weight
v = tf.Variable(tf.truncated_normal([1000, 1000], -0.1, 0.1))

# Encoder
h_previous = tf.zeros([1000, 1])
for t in range(N):
    # Current vector and its embedding
    xt = tf.reshape(tf.slice(X, [t, 0], [1, k]), [k, 1])
    e = tf.matmul(we, xt)
    # Reset calculation
    r = tf.sigmoid(tf.matmul(wr, e) + tf.matmul(ur, h_previous))
    # Update calculation
    z = tf.sigmoid(tf.matmul(wz, e) + tf.matmul(uz, h_previous))
    # Hidden-tilde calculation
    h_tilde = tf.tanh(tf.matmul(w, e) + tf.matmul(u, r * h_previous))
    # Hidden calculation
    one = tf.ones([1000])
    h = z * h_previous + (one - z) * h_tilde
    h_previous = h
c = tf.tanh(tf.matmul(v, h_previous))

Upvotes: 2

Related Questions