CODEWITHSUNDEEP
machine-learningneural-network
csz-carrot
csz-carrot

Reputation: 275

Meaning of parameters in RNNlib

I'm new to study recurrent neural networks and now confused by the parameters in RNNLib. Specifically, I don't understand the hidden Block, hidden size, input Block, subsample size and stuffs with mdl. In my experience, I just had input vectors, one lstm hidden layer and softmax output layer. Why does the block seem like a matrix?

Upvotes: 3

Views: 948

Answers (1)

Denis Tarasov
Denis Tarasov

Reputation: 1051

RNNLib implements a novel type of RNN, so-called "Multidimensional recurrent neural network". Following reference on RNNLib page explains that : Alex Graves, Santiago Fernández and Jürgen Schmidhuber.Multidimensional recurrent neural networks International Conference on Artificial Neural Networks, September 2007,Porto. This extension is designed for processing images, video and so on. As explained in the paper:

"The basic idea of MDRNNs is to replace the single recurrent connection found in standard RNNs with as many recurrent connections as there are dimensions in the data. During the forward pass, at each point in the data sequence, the hidden layer of the network receives both an external input and its own activations from one step back along all dimensions"

I think, that is the reason why you have ability to use multidimensional input. If you want to use RNNLib as usual one-dimensional RNN, just specify one dimension for input and LSTM block.

MDL stands for "Minimum Description length" cost function, used for approximation of Bayesian inference (a method for regularizing NN). If you want to use that, its best to read original references, provided on RNNLib website. Otherwise, I think, it can be just ignored.

Upvotes: 4

Related Questions