CODEWITHSUNDEEP
neural-networktheanodeep-learninglstm
Alex Gao
Alex Gao

Reputation: 2091

LSTM implementation using theano

I am using theano scan function for implementing LSTM(long short term memory), but I got error like 

ValueError: Please provide None as outputs_info for any output that does not feed back into scan (i.e. it behaves like a map)

I used scan like this 

p_c = T.matrix()
p_hidden_inputs = T.matrix()
out, updates = scan(step_fprop,
                sequences=model_inputs,
                outputs_info= [p_c, p_hidden_inputs],
                non_sequences=
                [
                Wxi, Whi, Wci, bi,
                Wxf, Whf, Wcf, bf,
                Wxc, Whc, bc,
                Wxo, Who, Wco, bo
                ],
                n_steps=n_steps,
                )

and step_fprop is defined as follows:

def step_fprop(inputs, p_c, p_hidden_inputs,
           Wxi, Whi, Wci, bi,
           Wxf, Whf, Wcf, bf,
           Wxc, Whc, bc,
           Wxo, Who, Wco, bo
           ):
"""
Construct the forward propagation
:return:
:rtype:
"""
# input gate
ig = T.nnet.sigmoid(T.dot(inputs, Wxi) +
                    T.dot(p_hidden_inputs, Whi) +
                    T.dot(p_c, Wci) +
                    bi)

# forget gate
fg = T.nnet.sigmoid(T.dot(inputs, Wxf) +
                    T.dot(p_hidden_inputs, Whf) +
                    T.dot(p_c, Wcf) +
                    bf)

# cell
cc= fg * p_c + ig * T.tanh(T.dot(inputs, Wxc) +
                                T.dot(p_hidden_inputs, Whc ) +
                                bc)

#output gate
og = T.nnet.sigmoid(T.dot(inputs, Wxo) +
                    T.dot(p_hidden_inputs,Who)  +
                    T.dot(p_c, Wco) +
                    bo)

#hidden state
hh = og * T.tanh(cc)

return hh

Anyone has any idea why I kept getting that kind of error.

Upvotes: 3

Views: 1809

Answers (1)

o1lo01ol1o
o1lo01ol1o

Reputation: 470

outputs_info expects values for each of the variables passed in the return of the step_fprop function. your code seems to only return the hidden state hh but ouputs_info expects two values whose initial state is defined by p_c and p_hidden_inputs

It looks like you'll need to return [_whatever_previous_lstm_state, hh] in the step_fprop function

Upvotes: 4

Related Questions