CODEWITHSUNDEEP
python-2.7audiotensorflow
Carlton Banks
Carlton Banks

Reputation: 365

Tensorflow RNN training won't execute?

I am currently trying to train this RNN network, but seem to be running into weird errors, which I am not able to decode.

The input to my rnn network is digital sampled audio files. As the audio file can be of different length, will the vector of the sampled audio also have different lengths.

The output or the target of the neural network is to recreate a 14 dimensional vector, containing certain information of the audio files. I've already know the target, by manually calculating it, but need to make it work with a neural network.

I am currently using tensorflow as framework.

My network setup looks like this: 

def last_relevant(output):
     max_length = int(output.get_shape()[1])
     relevant = tf.reduce_sum(tf.mul(output, tf.expand_dims(tf.one_hot(length, max_length), -1)), 1)
     return relevant

def length(sequence): ##Zero padding to fit the max lenght... Question whether that is a good idea.
    used = tf.sign(tf.reduce_max(tf.abs(sequence), reduction_indices=2))
    length = tf.reduce_sum(used, reduction_indices=1)
    length = tf.cast(length, tf.int32)
    return length

def cost(output, target):
    # Compute cross entropy for each frame.
    cross_entropy = target * tf.log(output)
    cross_entropy = -tf.reduce_sum(cross_entropy, reduction_indices=2)
    mask = tf.sign(tf.reduce_max(tf.abs(target), reduction_indices=2))
    cross_entropy *= mask
    # Average over actual sequence lengths.
    cross_entropy = tf.reduce_sum(cross_entropy, reduction_indices=1)
    cross_entropy /= tf.reduce_sum(mask, reduction_indices=1)
    return tf.reduce_mean(cross_entropy)
#----------------------------------------------------------------------#
#----------------------------Main--------------------------------------#
### Tensorflow neural network setup

batch_size = None
sequence_length_max = max_length
input_dimension=1

data = tf.placeholder(tf.float32,[batch_size,sequence_length_max,input_dimension])
target = tf.placeholder(tf.float32,[None,14])

num_hidden = 24 ## Hidden layer
cell = tf.nn.rnn_cell.LSTMCell(num_hidden,state_is_tuple=True)  ## Long short term memory

output, state = tf.nn.dynamic_rnn(cell, data, dtype=tf.float32,sequence_length = length(data))  ## Creates the Rnn skeleton

last = last_relevant(output)#tf.gather(val, int(val.get_shape()[0]) - 1) ## Appedning as last

weight = tf.Variable(tf.truncated_normal([num_hidden, int(target.get_shape()[1])]))
bias = tf.Variable(tf.constant(0.1, shape=[target.get_shape()[1]]))

prediction = tf.nn.softmax(tf.matmul(last, weight) + bias)

cross_entropy = cost(output,target)# How far am I from correct value?

optimizer = tf.train.AdamOptimizer() ## TensorflowOptimizer
minimize = optimizer.minimize(cross_entropy)

mistakes = tf.not_equal(tf.argmax(target, 1), tf.argmax(prediction, 1))
error = tf.reduce_mean(tf.cast(mistakes, tf.float32))

## Training ##

init_op = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init_op)

    batch_size = 1000
    no_of_batches = int(len(train_data)/batch_size)
    epoch = 5000
    for i in range(epoch):
        ptr = 0
        for j in range(no_of_batches):
            inp, out = train_data[ptr:ptr+batch_size], train_output[ptr:ptr+batch_size]
            ptr+=batch_size
            sess.run(minimize,{data: inp, target: out})
        print "Epoch - ",str(i)
    incorrect = sess.run(error,{data: test_data, target: test_output})
    print('Epoch {:2d} error {:3.1f}%'.format(i + 1, 100 * incorrect))
    sess.close()

The error seem to be the usage of the function last_relevant, which should take the output, and feed it back.

This is the error message: 

TypeError: Expected binary or unicode string, got <function length at 0x7f846594dde8>

Anyway to tell what could be wrong here?

Upvotes: 0

Views: 119

Answers (1)

Hakan
Hakan

Reputation: 618

I tried to build your code in my local. There is a fundamental mistake in the code which is that you call tf.one_hot but what you pass don't really fit with what is expected:

Read documentation here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/api_docs/python/functions_and_classes/shard6/tf.one_hot.md

tf.one_hot(indices, depth, on_value=None, off_value=None, axis=None, dtype=None, name=None)

However, you are passing a function pointer ("length" is a function in your code, I recommend naming your function in a meaningful manner by refraining yourself from using common keywords) instead of the first parameter.

For a wild guide, you can put your indices as first param (instead of my placeholder empty list) and it will be fixed

relevant = tf.reduce_sum(
      tf.mul(output, tf.expand_dims(tf.one_hot([], max_length), -1)), 1)

Upvotes: 1

Related Questions