Fully Connected layer incorrect feed

Question

Dragging some idea for building CNN from here, I want to build a convnet which comprises of two convolution layers, one fully connected layer(FCL) and softmax layer(SL). I couldn't understand defining the convolution operations to perform on FCL and back connected to SL.

In FCL, are the convolution operation performed in 1D where the input is flattened ? The weight for FCL are generated in 2D but how can I do the Conv operations if so ? because the matrix dimension dont match with the reshaped input and weights generated.( comparing VGGNET in detail column at the end). Even If I can do a 1xM and MxN conv operation the sizes of the matrix are missmatching where did I go wrong in FCL ?

Traceback (most recent call last):
File "D:/Lab_Project_Files/TF/Practice Files/basictest22.py", line 108, in <module>
   y = conv_net( x )
File "D:/Lab_Project_Files/TF/Practice Files/basictest22.py", line 93, in conv_net
   FClayer = tf.nn.relu(tf.add(tf.matmul(reshape,layer3_weights),layer3_biases))
ValueError: Shape must be rank 2 but is rank 1 for 'MatMul' (op: 'MatMul') with input shapes: [15360], [2240,64]

How to define the FCL ? I'm bit confused whether these operations apply on each and every image of the batch ?

My input parameters are

INPUT_WIDTH  = 16 # input image width
INPUT_HEIGHT = 12 # input image height
INPUT_DEPTH  = 1  # input image depth = 1 for monochrome
NUM_CLASSES  = 8  # output classes
BATCH_SIZE   = 5  # grouping batch for training 
# input output placeholders
x = tf.placeholder(tf.float32, [BATCH_SIZE, INPUT_WIDTH,INPUT_HEIGHT,INPUT_DEPTH ])
y_ = tf.placeholder(tf.float32, [BATCH_SIZE, NUM_CLASSES])

my trail code

def outputdetails(W1, H1,F, P, S):
# W1,W2 - width of input and output
# H1,H2 - height of input and output
# F     - size of the filter
# P     - padding
# S     - Stride
P = 0.00
W2 = int((W1 - F + 2*P)/S + 1)
H2 = int((H1 - F + 2*P)/S + 1)
return W2, H2

# CNN trail
def conv_net(x):
    # CONV1 layer
    FILTER_SIZE = 3   # applying 3x3 filter
    STRIDE = 1
    num_hidden = 64 # used for FCL as num of outputs
    NUM_CHANNELS = INPUT_DEPTH # input channels
    DEPTH = 16       # Output channels Apply 16 filters
    layer1_weights = tf.Variable(tf.random_normal([FILTER_SIZE,FILTER_SIZE,NUM_CHANNELS,DEPTH],stddev = 0.1))
    layer1_biases = tf.Variable(tf.zeros([DEPTH]))

    #CONV2 layer
    NUM_CHANNELS = 16
    DEPTH = 16
    layer2_weights = tf.Variable(tf.random_normal([FILTER_SIZE, FILTER_SIZE, NUM_CHANNELS, DEPTH], stddev=0.1))
    layer2_biases = tf.Variable(tf.zeros([DEPTH]))

    # Fully Connected layer
    # W1 - INPUT_WIDTH, H1 - INPUT_HEIGHT, F - FILTER_SIZE, S - STRIDE
    finalsize_width,finalsize_height = outputdetails(INPUT_WIDTH,INPUT_HEIGHT,FILTER_SIZE,1,STRIDE)
    layer3_weights = tf.Variable(
    tf.truncated_normal([finalsize_width * finalsize_height * DEPTH, num_hidden], stddev=0.1))
    layer3_biases = tf.Variable(tf.constant(1.0, shape=[num_hidden]))
    # softmax layer
    Outlayer_weights = tf.Variable(tf.random_normal([num_hidden, NUM_CLASSES], stddev=0.1))
    Outlayer_biases = tf.Variable(tf.constant(1.0,shape = [NUM_CLASSES]))

    conv1 = tf.nn.relu(tf.add(tf.nn.conv2d(x,layer1_weights,strides = [1,1,1,1],padding='SAME'),layer1_biases))
    conv2 = tf.nn.relu(tf.add(tf.nn.conv2d(conv1, layer2_weights, strides=[1, 1, 1, 1], padding='SAME'), layer2_biases))
    shape = conv2.get_shape().as_list()
    reshape = tf.reshape(conv2,[shape[0]*shape[1]*shape[2]*shape[3]])
    FClayer = tf.nn.relu(tf.add(tf.matmul(reshape,layer3_weights),layer3_biases))
    out = tf.add(tf.matmul(FClayer, Outlayer_weights), Outlayer_biases)
    return out

Files if required source file

classes

data

Answer 1

Change this

reshape = tf.reshape(conv2,[shape[0]*shape[1]*shape[2]*shape[3]])

to this

reshape = tf.reshape(conv2,[shape[0],shape[1]*shape[2]*shape[3]])

matmul can work with a batch dimension which you are destroying.