CODEWITHSUNDEEP
pythontensorflowneural-network
Salman Shaukat
Salman Shaukat

Reputation: 343

How to determine weight dimension for input tensor of Rank-3?

I'm trying to design an Autoencoder for activity classification for 3-channel input (Tri-axial accelerometer data).

The input tensor is of shape [None,200,3] ([Batch size, window size, number of channels]) and in the first layer, I want to simply reduce the dimension of input layer to [None,150,3]. Here is the code for creating placeholders and the first layer:

import tensorflow as tf

def denseLayer(inputVal,weight,bias):
    return tf.nn.relu((tf.matmul(inputVal,weight)+bias))


x = tf.placeholder(dtype=tf.float32,shape=[None,200,3]) #Input tensor
wIn = tf.get_variable(name='wIn',initializer=tf.truncated_normal(stddev=0.1,dtype=tf.float32,shape=[200,150]))

bIn = tf.get_variable(name='bIn',initializer=tf.constant(value = 0.1,shape=[150,3],dtype=tf.float32))


firstLayer = denseLayer(x,weight=wIn,bias=bIn)

This code will, of course, result in an error (due to the difference in rank between x and wIn) and i am unable to determine the shape of wIn variable to get the desired shape of firstLayer that is [None,150,3].

Here is how the final network should look (simplified version with lesser layers): Autoencoder

Upvotes: 1

Views: 297

Answers (1)

javidcf
javidcf

Reputation: 59731

I think this does what you want:

import tensorflow as tf

def denseLayer(inputVal, weight, bias):
    # Each input "channel" uses the corresponding set of weights
    value = tf.einsum('nic,ijc->njc', inputVal, weight) + bias
    return tf.nn.relu(value)
#Input tensor
x = tf.placeholder(dtype=tf.float32, shape=[None, 200, 3])
# Weights and biases have three "channels" each
wIn = tf.get_variable(name='wIn',
                      shape=[200, 150, 3],
                      initializer=tf.truncated_normal_initializer(stddev=0.1))
bIn = tf.get_variable(name='bIn',
                      shape=[150, 3],
                      initializer=tf.constant_initializer(value=0.1))
firstLayer = denseLayer(x, weight=wIn, bias=bIn)
print(firstLayer)
# Tensor("Relu:0", shape=(?, 150, 3), dtype=float32)

Here wIn can be seen as three sets of [200, 150] parameters that are applied to each input channel. I think tf.einsum is the easiest way to implement that in this case.

Upvotes: 1

Related Questions