CODEWITHSUNDEEP
pythontensorflowautoencoderrbm
z991
z991

Reputation: 713

Implementation of Sparse autoencoder by tensorflow

I am trying to implement simple autoencoder like below.

The number of input features are 2, and I want to build sparse autoencoder for dimension reduction to feature 1. I selected the number of nodes are 2(input), 8(hidden), 1(reduced feature), 8(hidden), 2(output) to add some more complexity than using only (2, 1, 2) nodes. The number of samples N is around 10000. 'DATA' is a just a 2x10000 matrix containing integer values.

import tensorflow as tf

x = tf.placeholder(shape=[None, 2])
w1 = tf.Variable(tf.random_normal(shape=[2, 8]))
w2 = tf.Variable(tf.random_normal(shape=[8, 1]))
h1 = tf.nn.relu(tf.matmul(x, w1))
encoded = tf.matmul(h1, w2)
h2 = tf.nn.relu(encoded)
h3 = tf.nn.relu(tf.matmul(h2, tf.transpose(w2)))
y = tf.matmul(h3, tf.transpose(w1))
mse = tf.reduce_mean(tf.squared_difference(x, y))
optimizer = 
tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(mse)
sess = tf.Session()
sess.run(init)
fd = {x: DATA}
loss_value, reduced_feature = sess.run([mse, encoded], feed_dict=fd)

I have 2 questions with the implementation, as the result was quite different as I expected.

  1. Is this implementation correct? Will the variable 'reduced_feature' show the reduced feature(1d feature) from 2 feature inputs?

  2. Should I add some sparsity condition if I want to use more hidden nodes than input? If yes, can you show some sample code for this task?

Upvotes: 0

Views: 531

Answers (0)

Related Questions