CODEWITHSUNDEEP
pythontensorflowkeras
vahid
vahid

Reputation: 11

tensorflow backend in keras for video

I am using keras with tensorflow backend,my data is video.

So now I have a problem with data dimension, I know first channels is for theano and last channels for tf. I have problem where to put number of frames.

  1. Is this true? (number of samples,number of frames,img_rows,img_cols,channels) -- for example:(390,25,48,48,1)

  2. or is this true? (number of samples,img_rows,img_cols,channels,number of frames) -- for example like this: (360,48,48,1,25)

Which one is true for keras with tf backend for video data? 1 or 2?
Thanks a lot

Upvotes: 1

Views: 229

Answers (1)

Daniel Möller
Daniel Möller

Reputation: 86630

Keras by itself defines its data format as channels_last. There is no need to worry about whether you're using theano or tensorflow. (You can change this either in convolutional layers or in the default settings at the file <user>\.keras\keras.json)

But this is valid mainly for images and other convolutional layers: (img_row,img_cols,channels)

As far as I know, there is no predefined format for video.

  • When you are using recurrent layers, you will have (batchSize,timeSteps,featuresPerStep)
  • When using 2D convolutions: (batchSize, rows, columns, channels)
  • When using 1D convolutions: (batchSize, length, channels)

There is the possibility to use a layer wrapper called TimeDistributed, which will allow non recurrent layers to receive inputs with the additional timeSteps dimension just after the batch dimension: (batchSize,timeSteps, ...other dimensions....) -- It will replicate the wrapped layer for each time step.

So, a 2D convolutional layer with the time distributed wrapper would take inputs as: (batchSize,timeSteps,rows,columns,channels)

Upvotes: 1

Related Questions