CODEWITHSUNDEEP
tensorflowpytorchrecurrent-neural-network
Beta
Beta

Reputation: 1756

Understanding the Input Parameters in RNN

I'm having a hard time to understand the different "jargons" used in RNN. They are the following: batch_size, time_steps, inputs and instances.

Let me go through my understanding of each input parameters & please correct me where I'm wrong. Suppose I've got a sequence of numbers and I want to predict the next number. The numbers are the following: [1,2,3,4,5,....,100]

time_steps: This parameter means how far RNN will look into past before it predicts the future. For simplicity, I want to predict 1 number ahead. And want to do after I see 10 numbers in the past. So, in this case, time_steps will be 10.

inputs: These are the values at each time_steps. In first time_step (t) the inputs are t0: [1]

t1: [2]

.
.
.

t10: [10]`

batch_size: This helps in efficient computation of RNN model. Suppose my batch_size is 2. In that case, at time_step 2, the RNN input will be

t0: [1]
t0: [11]

Then what's the usage of instances? E.g. in this post, instances have been used. And there are multiple cases where instances are used. Is it means each loop over batch? E.g. there are 5 batches, each of size 2. Then there will be 5 instances.

Please help me correct my understanding.

Thanks!

Upvotes: 1

Views: 524

Answers (2)

Wasi Ahmad
Wasi Ahmad

Reputation: 37761

batch_size

Batch size, in general, represents the size of the mini-batches constructed from the experimental dataset. Since in deep learning, we are required to do a lot of computations, it is better if we consider mini-batch operations because GPU usage will be worth then.

time_steps

Since RNN takes sequential inputs, index of each element in the input sequence can be referred as a time step of that sequence. For example, if [1,2,3,4,5,....,100] is a sequence, index of each element in the sequence is a time step.

inputs

The term inputs has a broader meaning, so I am not sure if my definition is correct. According to my understanding, inputs to an RNN refers to individual inputs provided to RNN at each time step. For example, in [1,2,3,4,5,....,100], each element is an input to the RNN at a particular time step.

But in an abstract way, if someone asks, what is the input of your deep neural model? You can say, it is English sentences or images or audio clips or videos etc. In short, the meaning of the term inputs depends on the context.

instances

Instances, in general, refers to a training/dev/test example in the dataset. For example, the sequence: [1,2,3,4,5,....,100] can be a training instance in your dataset.

Hope this helps!

Upvotes: 1

shivam13juna
shivam13juna

Reputation: 347

Alright pal, you did good learning those concepts. I had a hard time learning those correctly. Everything you know seems to be in order and as for "instances". They're basically a set of data. There's no fixed term of usage of "instances" in a deep learning community. Some people use it for referring for a different set of data or batches of data. I rarely hear it in papers.

Upvotes: 0

Related Questions