Need Help in creating ML model with TensorFlow 2.0

Question

I'm new to machine learning and python. I have successfully trained a linear model with TensorFlow 2. I have standardized the input with the help of preprocessing.scale() function available under sklearn.

The issue here is that my predicated output for my test data varies greatly from my actual target values. I have a single layer model and using sgd as an optimizer, mean_squared_error as my loss function and accuracy as my metrics.

Training data contains 4 parameters and one target (273.39,420.12,91.84,57.41,1778.12). Last column is target.

Plot of my Models predicted output vs actual target is not linear at all as shown below.

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
from sklearn import preprocessing

# Extract Data from CSV

train_csv_data = np.loadtxt('train_data.csv',delimiter=',')
test_csv_data = np.loadtxt('test_data.csv',delimiter=',')

train_unscaled_input_all = train_csv_data[:,0:-1]
train_target_all = train_csv_data[:,-1]

# Shuffling the input

shuffled_indices = np.arange(train_unscaled_input_all.shape[0])
np.random.shuffle(shuffled_indices)

shuffled_inputs = train_unscaled_input_all[shuffled_indices]
shuffled_targets = train_target_all[shuffled_indices]


# Split date into train , validation and test

total_count = shuffled_inputs.shape[0]
train_data_count = int (0.8*total_count)
validation_data_count = int (0.1*total_count)
test_data_count = total_count - train_data_count - validation_data_count

train_inputs = shuffled_inputs[:train_data_count]
train_targets = shuffled_targets[:train_data_count]

validation_inputs = shuffled_inputs[train_data_count:train_data_count+validation_data_count]
validation_targets =         
    shuffled_targets[train_data_count:train_data_count+validation_data_count]

test_inputs = shuffled_inputs[train_data_count+validation_data_count:]
test_targets = shuffled_targets[train_data_count+validation_data_count:]



# Standardize the Inputs

train_inputs = preprocessing.scale(train_inputs)
train_targets = preprocessing.scale(train_targets)

validation_inputs = preprocessing.scale(validation_inputs)
validation_targets = preprocessing.scale(validation_targets)


# Save 3 Datasets in .npz
np.savez('train',inputs = train_inputs, targets = train_targets)
np.savez('validation',inputs = validation_inputs, targets = validation_targets)
np.savez('test',inputs = test_inputs, targets = test_targets)



train_npz = np.load('train.npz')
train_inputs = train_npz['inputs']
train_targets = train_npz['targets']

validation_npz = np.load('validation.npz')
validation_inputs = validation_npz['inputs']
validation_targets = validation_npz['targets']

test_npz = np.load('test.npz')
test_inputs = test_npz['inputs']
test_targets = test_npz['targets']

# Model Creation


output_size = 1

model = tf.keras.Sequential([                            
                          tf.keras.layers.Dense(output_size)
                      ])
model.compile(optimizer='sgd', loss='mean_squared_error',metrics=['accuracy'])

max_epochs = 50
model.fit(train_inputs,train_targets, epochs = max_epochs, validation_data= . 
   (validation_inputs,validation_targets), verbose=1 )
predicted_outputs = (model.predict_on_batch(test_inputs))

plt.plot(np.squeeze(predicted_outputs),np.squeeze(test_targets))
plt.xlabel('Output')
plt.ylabel('Target')
plt.show()

My Training and validation accuracies are zero throughout the training process. Obviously I'm missing something here. It would be great if you point out what I'm missing here in training this linear model.

UPDATE : Training Model

Answer 1

Thank you for attaching the data. I now have a handle on your problem.

I could replicate the problem with your code.

There were following issues with your existing code

1. You do preprocessing.scale for train and validation but you don't do it for test dataset. Thus for test the values are very different than the values which were used for training. It is not going to work effectively. So you will have to preprocess your test dataset as well.
2. You don't have any hidden unit and as @venkatakrishnan has already pointed out it is like a perceptron. You need to add hidden unit(s) to show some improvement.
3. In plt you are using plt.plot. I suggest you use plt.scatter as the picture will be more meaningful and less confusing.
4. You were using 'accuracy' as a metrics. Since you are not doing classification rather regression, I think 'mae' will be a better metrics.

When you incorporate those changes, you can get a code something like below (with minimal changes).

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
from sklearn import preprocessing

# Extract Data from CSV

train_csv_data = np.loadtxt('train_data.csv',delimiter=',')
test_csv_data = np.loadtxt('test_data.csv',delimiter=',')

train_unscaled_input_all = train_csv_data[:,0:-1]
train_target_all = train_csv_data[:,-1]

# Shuffling the input

shuffled_indices = np.arange(train_unscaled_input_all.shape[0])
np.random.shuffle(shuffled_indices)

shuffled_inputs = train_unscaled_input_all[shuffled_indices]
shuffled_targets = train_target_all[shuffled_indices]


# Split date into train , validation and test

total_count = shuffled_inputs.shape[0]
train_data_count = int (0.8*total_count)
validation_data_count = int (0.1*total_count)
test_data_count = total_count - train_data_count - validation_data_count

train_inputs = shuffled_inputs[:train_data_count]
train_targets = shuffled_targets[:train_data_count]

validation_inputs = shuffled_inputs[train_data_count:train_data_count+validation_data_count]
validation_targets = shuffled_targets[train_data_count:train_data_count+validation_data_count]

test_inputs = shuffled_inputs[train_data_count+validation_data_count:]
test_targets = shuffled_targets[train_data_count+validation_data_count:]


# Standardize the Inputs

train_inputs = preprocessing.scale(train_inputs)
train_targets = preprocessing.scale(train_targets)

validation_inputs = preprocessing.scale(validation_inputs)
validation_targets = preprocessing.scale(validation_targets)

test_inputs = preprocessing.scale(test_inputs)
test_targets = preprocessing.scale(test_targets)

# Save 3 Datasets in .npz
np.savez('train',inputs = train_inputs, targets = train_targets)
np.savez('validation',inputs = validation_inputs, targets = validation_targets)
np.savez('test',inputs = test_inputs, targets = test_targets)

train_npz = np.load('train.npz')
train_inputs = train_npz['inputs']
train_targets = train_npz['targets']

validation_npz = np.load('validation.npz')
validation_inputs = validation_npz['inputs']
validation_targets = validation_npz['targets']

test_npz = np.load('test.npz')
test_inputs = test_npz['inputs']
test_targets = test_npz['targets']

# Model Creation


output_size = 1

model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(1000, input_dim=4, activation='linear'))
model.add(tf.keras.layers.Dense(1, activation='linear'))
model.compile(optimizer='sgd', loss='mean_squared_error',metrics=['mae'])

max_epochs = 50
model.fit(train_inputs,train_targets, epochs = max_epochs, validation_data= (validation_inputs,validation_targets), verbose=1 )
predicted_outputs = (model.predict_on_batch(test_inputs))

plt.scatter(np.squeeze(predicted_outputs),np.squeeze(test_targets))
plt.xlabel('Output')
plt.ylabel('Target')
plt.show()

This gives the following plot, which is nice and linear and there is a very nice correlation between predicted value and actual preprocessed value. You will have to do some kind of inverse transform to bring them to original values.

During the training loss and mae decreased to certain extent

Epoch 1/50
1600/1600 [==============================] - 0s 98us/sample - loss: 0.1561 - mean_absolute_error: 0.2963 - val_loss: 0.0811 - val_mean_absolute_error: 0.2270
Epoch 2/50
1600/1600 [==============================] - 0s 43us/sample - loss: 0.0797 - mean_absolute_error: 0.2230 - val_loss: 0.0779 - val_mean_absolute_error: 0.2210
Epoch 3/50
1600/1600 [==============================] - 0s 42us/sample - loss: 0.0756 - mean_absolute_error: 0.2170 - val_loss: 0.0767 - val_mean_absolute_error: 0.2194
Epoch 4/50
1600/1600 [==============================] - 0s 46us/sample - loss: 0.0735 - mean_absolute_error: 0.2140 - val_loss: 0.0775 - val_mean_absolute_error: 0.2226
Epoch 5/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0732 - mean_absolute_error: 0.2137 - val_loss: 0.0791 - val_mean_absolute_error: 0.2263
Epoch 6/50
1600/1600 [==============================] - 0s 44us/sample - loss: 0.0727 - mean_absolute_error: 0.2126 - val_loss: 0.0774 - val_mean_absolute_error: 0.2223
Epoch 7/50
1600/1600 [==============================] - 0s 41us/sample - loss: 0.0725 - mean_absolute_error: 0.2125 - val_loss: 0.0786 - val_mean_absolute_error: 0.2258
Epoch 8/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0724 - mean_absolute_error: 0.2126 - val_loss: 0.0788 - val_mean_absolute_error: 0.2251
Epoch 9/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0724 - mean_absolute_error: 0.2124 - val_loss: 0.0782 - val_mean_absolute_error: 0.2227
Epoch 10/50
1600/1600 [==============================] - 0s 44us/sample - loss: 0.0724 - mean_absolute_error: 0.2124 - val_loss: 0.0789 - val_mean_absolute_error: 0.2257
Epoch 11/50
1600/1600 [==============================] - 0s 46us/sample - loss: 0.0728 - mean_absolute_error: 0.2128 - val_loss: 0.0793 - val_mean_absolute_error: 0.2262
Epoch 12/50
1600/1600 [==============================] - 0s 47us/sample - loss: 0.0722 - mean_absolute_error: 0.2118 - val_loss: 0.0796 - val_mean_absolute_error: 0.2268
Epoch 13/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0725 - mean_absolute_error: 0.2127 - val_loss: 0.0785 - val_mean_absolute_error: 0.2247
Epoch 14/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0723 - mean_absolute_error: 0.2124 - val_loss: 0.0789 - val_mean_absolute_error: 0.2243
Epoch 15/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0722 - mean_absolute_error: 0.2117 - val_loss: 0.0814 - val_mean_absolute_error: 0.2305
Epoch 16/50
1600/1600 [==============================] - 0s 47us/sample - loss: 0.0725 - mean_absolute_error: 0.2123 - val_loss: 0.0784 - val_mean_absolute_error: 0.2252
Epoch 17/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0726 - mean_absolute_error: 0.2125 - val_loss: 0.0786 - val_mean_absolute_error: 0.2264
Epoch 18/50
1600/1600 [==============================] - 0s 44us/sample - loss: 0.0725 - mean_absolute_error: 0.2122 - val_loss: 0.0785 - val_mean_absolute_error: 0.2252
Epoch 19/50
1600/1600 [==============================] - 0s 48us/sample - loss: 0.0722 - mean_absolute_error: 0.2120 - val_loss: 0.0784 - val_mean_absolute_error: 0.2253
Epoch 20/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0727 - mean_absolute_error: 0.2123 - val_loss: 0.0785 - val_mean_absolute_error: 0.2255
Epoch 21/50
1600/1600 [==============================] - 0s 44us/sample - loss: 0.0724 - mean_absolute_error: 0.2121 - val_loss: 0.0785 - val_mean_absolute_error: 0.2225
Epoch 22/50
1600/1600 [==============================] - 0s 48us/sample - loss: 0.0722 - mean_absolute_error: 0.2118 - val_loss: 0.0790 - val_mean_absolute_error: 0.2240
Epoch 23/50
1600/1600 [==============================] - 0s 44us/sample - loss: 0.0723 - mean_absolute_error: 0.2120 - val_loss: 0.0792 - val_mean_absolute_error: 0.2273
Epoch 24/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0723 - mean_absolute_error: 0.2119 - val_loss: 0.0804 - val_mean_absolute_error: 0.2286
Epoch 25/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0725 - mean_absolute_error: 0.2121 - val_loss: 0.0802 - val_mean_absolute_error: 0.2268
Epoch 26/50
1600/1600 [==============================] - 0s 47us/sample - loss: 0.0724 - mean_absolute_error: 0.2119 - val_loss: 0.0792 - val_mean_absolute_error: 0.2266
Epoch 27/50
1600/1600 [==============================] - 0s 43us/sample - loss: 0.0721 - mean_absolute_error: 0.2116 - val_loss: 0.0821 - val_mean_absolute_error: 0.2319
Epoch 28/50
1600/1600 [==============================] - 0s 44us/sample - loss: 0.0726 - mean_absolute_error: 0.2128 - val_loss: 0.0784 - val_mean_absolute_error: 0.2239
Epoch 29/50
1600/1600 [==============================] - 0s 42us/sample - loss: 0.0725 - mean_absolute_error: 0.2125 - val_loss: 0.0783 - val_mean_absolute_error: 0.2228
Epoch 30/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0725 - mean_absolute_error: 0.2120 - val_loss: 0.0785 - val_mean_absolute_error: 0.2253
Epoch 31/50
1600/1600 [==============================] - 0s 46us/sample - loss: 0.0723 - mean_absolute_error: 0.2118 - val_loss: 0.0784 - val_mean_absolute_error: 0.2251
Epoch 32/50
1600/1600 [==============================] - 0s 41us/sample - loss: 0.0721 - mean_absolute_error: 0.2116 - val_loss: 0.0788 - val_mean_absolute_error: 0.2235
Epoch 33/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0724 - mean_absolute_error: 0.2122 - val_loss: 0.0786 - val_mean_absolute_error: 0.2247
Epoch 34/50
1600/1600 [==============================] - 0s 44us/sample - loss: 0.0725 - mean_absolute_error: 0.2124 - val_loss: 0.0787 - val_mean_absolute_error: 0.2248
Epoch 35/50
1600/1600 [==============================] - 0s 47us/sample - loss: 0.0722 - mean_absolute_error: 0.2116 - val_loss: 0.0791 - val_mean_absolute_error: 0.2267
Epoch 36/50
1600/1600 [==============================] - 0s 43us/sample - loss: 0.0722 - mean_absolute_error: 0.2120 - val_loss: 0.0793 - val_mean_absolute_error: 0.2263
Epoch 37/50
1600/1600 [==============================] - 0s 44us/sample - loss: 0.0727 - mean_absolute_error: 0.2129 - val_loss: 0.0787 - val_mean_absolute_error: 0.2255
Epoch 38/50
1600/1600 [==============================] - 0s 44us/sample - loss: 0.0723 - mean_absolute_error: 0.2118 - val_loss: 0.0800 - val_mean_absolute_error: 0.2273
Epoch 39/50
1600/1600 [==============================] - 0s 42us/sample - loss: 0.0722 - mean_absolute_error: 0.2117 - val_loss: 0.0801 - val_mean_absolute_error: 0.2276
Epoch 40/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0722 - mean_absolute_error: 0.2112 - val_loss: 0.0795 - val_mean_absolute_error: 0.2273
Epoch 41/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0721 - mean_absolute_error: 0.2120 - val_loss: 0.0784 - val_mean_absolute_error: 0.2243
Epoch 42/50
1600/1600 [==============================] - 0s 48us/sample - loss: 0.0725 - mean_absolute_error: 0.2122 - val_loss: 0.0785 - val_mean_absolute_error: 0.2258
Epoch 43/50
1600/1600 [==============================] - 0s 43us/sample - loss: 0.0726 - mean_absolute_error: 0.2126 - val_loss: 0.0784 - val_mean_absolute_error: 0.2256
Epoch 44/50
1600/1600 [==============================] - 0s 44us/sample - loss: 0.0723 - mean_absolute_error: 0.2119 - val_loss: 0.0786 - val_mean_absolute_error: 0.2246
Epoch 45/50
1600/1600 [==============================] - 0s 45us/sample - loss: 0.0723 - mean_absolute_error: 0.2120 - val_loss: 0.0784 - val_mean_absolute_error: 0.2242
Epoch 46/50
1600/1600 [==============================] - 0s 47us/sample - loss: 0.0722 - mean_absolute_error: 0.2116 - val_loss: 0.0793 - val_mean_absolute_error: 0.2270
Epoch 47/50
1600/1600 [==============================] - 0s 41us/sample - loss: 0.0725 - mean_absolute_error: 0.2125 - val_loss: 0.0782 - val_mean_absolute_error: 0.2240
Epoch 48/50
1600/1600 [==============================] - 0s 43us/sample - loss: 0.0723 - mean_absolute_error: 0.2120 - val_loss: 0.0793 - val_mean_absolute_error: 0.2277
Epoch 49/50
1600/1600 [==============================] - 0s 42us/sample - loss: 0.0723 - mean_absolute_error: 0.2123 - val_loss: 0.0783 - val_mean_absolute_error: 0.2246
Epoch 50/50
1600/1600 [==============================] - 0s 48us/sample - loss: 0.0723 - mean_absolute_error: 0.2117 - val_loss: 0.0788 - val_mean_absolute_error: 0.2269

Please let me know if this is what you expected or my answer is off.