CODEWITHSUNDEEP
pythontensorflownon-linear-regression
yanniknelson
yanniknelson

Reputation: 31

Tensorflow sigmoid regression stays linear

I'm trying to make a simple neural net fit a simple function using tensor flow, I know that the structure and parameters I'm using work as I've achieved this in MatLab, but I need to port this over to another language (currently Python, but later c++). As such I'm trying to find a good neural network library, I thought that would be TensorFlow, but it's proving very finicky. Here are the important bits of the code

import numpy as np
import tensorflow as tf
from tensorflow.keras import layers
import matplotlib.pyplot as plt

f1 = lambda x: ((x < .5) * np.power(x, 2) + (x > .5) * x) * 2 -1
x = np.linspace(-1, 1, 180).reshape(-1,1)
y = f1(x).reshape(-1, 1)
model = keras.models.Sequential()
model.add(layers.Dense(100, activation=tf.keras.activations.sigmoid, input_shape=(1,)))
model.add(layers.Dense(1, activation=tf.keras.activations.linear))
model.compile(loss=tf.keras.losses.mean_squared_error, optimizer=tf.keras.optimizers.SGD(0.001), metrics=[tf.keras.losses.mean_squared_error])
model.fit(x, y, batch_size=1, epochs=3)
xtest = np.linspace(-1, 1, 100).reshape(-1, 1)
ytest = model.predict(xtest)

plt.scatter(x, y)
plt.plot(xtest, ytest)
plt.show()

This when the expected plot and predictions are plotted it results in This Plot where the dots are the expected function and the solid line is the prediction I'm not sure what I'm doing wrong.

The network has to be a layer of 100 sigmoid activated neurons followed by a linear output layer, even if I change the number of epochs and batch sizes the network still trains linear. Any help would be greatly appreciated

Upvotes: 0

Views: 164

Answers (1)

Vishwas Chepuri
Vishwas Chepuri

Reputation: 731

You may try changing activation and increasing training time, I changed optimizer to Adam, increased training epochs to 30 and got this result.
result image

Code :

import numpy as np
import tensorflow as tf
from tensorflow.keras import layers
import matplotlib.pyplot as plt

f1 = lambda x: ((x < .5) * np.power(x, 2) + (x > .5) * x) * 2 -1
x = np.linspace(-1, 1, 180).reshape(-1,1)
y = f1(x).reshape(-1, 1)
model = keras.models.Sequential()
model.add(layers.Dense(100, activation=tf.keras.activations.sigmoid, input_shape=(1,)))
model.add(layers.Dense(1, activation=tf.keras.activations.linear))
model.compile(loss=tf.keras.losses.mean_squared_error, 
              optimizer=tf.keras.optimizers.Adam(0.01), 
              metrics=[tf.keras.losses.mean_squared_error])
model.fit(x, y, batch_size=1, epochs=30)
xtest = np.linspace(-1, 1, 100).reshape(-1, 1)
ytest = model.predict(xtest)

plt.scatter(x, y)
plt.plot(xtest, ytest)
plt.show()

Upvotes: 1

Related Questions