In tensorflow or keras, How can we approximate a polynomial function like y=x^2?

Question

Im trying to approximate the function y=x^2, but the results are completely wrong. If i predict 2 the y number is -164455.89.

import tensorflow as tf
import numpy as np
from tensorflow import keras
xs = []
ys = []
for i in range(0,1000):
    xs.append(i)
    ys.append(i*i)

   
model = tf.keras.models.Sequential()

model.add(tf.keras.layers.Dense(units=1,input_shape=[1]))
model.add(tf.keras.layers.Dense(128,))
model.add(tf.keras.layers.Dense(128,))

model.add(tf.keras.layers.Dense(1,))
model.compile(optimizer='adam',loss = 'mean_squared_error')

model.fit(xs,ys,epochs=1000)
model.predict([2])

Answer 1

You need to consider:

1. Transform your xs and ys to range (0, 1). You can use sklearn.preprocessing.MinMaxScaler.
2. You can use numpy.linspace for creating more numbers for the range that you want.
3. Use a larger network and more neurons in the first Dense layer for getting better results.
4. Make sure to use activation=relu in the all Dense layer except the last layer.

Example Code: (for model.predict([[2]]) -> we get [[4.008407]])

from sklearn.preprocessing import MinMaxScaler
import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np


xs = np.linspace(-200,200,5000).reshape(-1,1)
ys = xs**2


xs = MinMaxScaler().fit_transform(xs)
ys = MinMaxScaler().fit_transform(ys)


model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(units=128,input_shape=(1,), activation='relu'))
model.add(tf.keras.layers.Dense(64, activation='relu'))
model.add(tf.keras.layers.Dense(32, activation='relu'))
model.add(tf.keras.layers.Dense(16, activation='relu'))
model.add(tf.keras.layers.Dense(1,))
model.compile(optimizer='adam',loss = 'mae')

model.fit(xs,ys,epochs=300, batch_size=64)
y_pred = model.predict(([[2]]))
print((y_pred))

y_pred = model.predict(xs, batch_size=16)
plt.plot(xs.reshape(-1), y_pred, 'r')
plt.plot(xs.reshape(-1), ys, 'b:')
plt.show()

Output:

Epoch 1/300
79/79 [==============================] - 1s 3ms/step - loss: 0.2474
Epoch 2/300
79/79 [==============================] - 0s 2ms/step - loss: 0.1370
Epoch 3/300
79/79 [==============================] - 0s 2ms/step - loss: 0.0238
Epoch 4/300
79/79 [==============================] - 0s 2ms/step - loss: 0.0077
...
Epoch 295/300
79/79 [==============================] - 0s 2ms/step - loss: 0.0032
Epoch 296/300
79/79 [==============================] - 0s 2ms/step - loss: 0.0028
Epoch 297/300
79/79 [==============================] - 0s 2ms/step - loss: 0.0035
Epoch 298/300
79/79 [==============================] - 0s 2ms/step - loss: 0.0057
Epoch 299/300
79/79 [==============================] - 0s 2ms/step - loss: 0.0024
Epoch 300/300
79/79 [==============================] - 0s 2ms/step - loss: 0.0028
[[4.008407]]