CODEWITHSUNDEEP
pythontensorflowneural-networkregression
sandboxj
sandboxj

Reputation: 1254

Nonlinear input to output mapping (undefined range) using tensorflow

I have an array of 1D input data (30,1). I m trying to map this to output data (30,1) (with noise). I have plotted the data and it is definitely non-linear and continuous.

I want to train a neural network to reproduce this mapping. I am currently trying to complete this task using tensorflow.

My problem right now is that the output data is in an undefined range (e.g. -2.74230671e+01, 1.00000000e+03, 6.34566772e+02 etc), and non-linear tensorflow activation functions seem to all between -1 and 1? https://www.tensorflow.org/versions/r0.12/api_docs/python/nn/activation_functions_

I am rather new to tensorflow etc, so my question is, how do I approach this problem?

I thought I could mean-normalize the data, but since I don't actually know the range of the output values (possibly unbounded).

Is this possible using tensorflow functions or will I need to build my own? The approach I am using is below, where I tried different functions for tf.nn.relu:

tf_x = tf.placeholder(tf.float32, x.shape)     # input x
tf_y = tf.placeholder(tf.float32, y.shape)     # output y

# neural network layers
l1 = tf.layers.dense(tf_x, 50, tf.nn.relu)     # tried different activation functions here
output = tf.layers.dense(l1, 1)                # tried here too

loss = tf.losses.mean_squared_error(tf_y, output)   
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.05)
train_op = optimizer.minimize(loss)

#train
for step in range(30):
_, l, pred = sess.run([train_op, loss, output], {tf_x: x, tf_y: y})
print(sess.run(loss, feed_dict={tf_x: x, tf_y: y}))

Upvotes: 0

Views: 349

Answers (1)

jkschin
jkschin

Reputation: 5844

You definitely have to normalize your data for it to work and it does not necessarily have to be in the range [-1, 1].

Take a Computer Vision (CV) problem as an example. What some papers do is simply divide by 255.0. Other papers, compute the mean and standard_deviation of each RGB channel from all the images. To normalize the images, we simply do (x-mu)/sigma over each channel.

Since your data is unbounded like what you said, then we can't simply divide by a scalar. Perhaps the best approach is to normalize based on the data statistics. Specific to your case, you could perhaps find the mean and standard_deviation of each of your 30 dimensions.

This post is more detailed and will potentially help you.

Upvotes: 1

Related Questions