How to predict new data with a trained neural network (Tensorflow 2.0, regression analysis)?

Question

I am new to machine learning and using Tensorflow. I have trained a neural network for regression following the tutorial on the Tensorflow website. I have 3 input columns and 2 output columns which I have marked as "labels". The network seemingly predicts data fine when using testing data, but when I try to predict data outside the testing and training set, by importing a file with 3 input columns only, it gives me an error saying "expected dense_input to have shape (5,) but got array with shape (3,)". I understand its because the model was trained on a 5 column data set, but I want to predict unknown values from the model (once trained) and don't know the output. How do I predict the results when I only know the input (3 columns)? And if I must know the output as well (which I am sure I mustn't), what's the point of this regression analysis?

My data looks like this:

And I am trying to make the neural network perform in this manner:

I want to predict outputs for which I don't have the data (say, RE = 25, BR=0.5, PR=0.25), and I can't use linear regression because the relation between inputs and outputs is not linear. I have tried predicting the input as a 5 column file after training, with the last two columns being junk (zeros), but the model tries to predict the zeros. As I understand, once the model has been trained, the weights and biases should be fixed and the model should, regardless of what is in the last two (output) columns, give me the expected output. What am I doing wrong? Any help is appreciated. I have uploaded the files used in the code here https://drive.google.com/open?id=1HoMID_razNHcXEGIgvnL8WG3H5WRTl3B . Additionally, sometimes my MSE (mean square error) converges, other times it does not. I suspect it may have something to do with randomly feeding the data to the model, although I am not sure.

import tensorflow as tf
from tensorflow import keras
import numpy as np 
import matplotlib.pyplot as plt 
import pandas as pd 
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
import pickle

column_names = ['Reynolds Number', 'Blockage Ratio', 'Prandtl Number', 'Nusselt Number', 'Drag Coefficient']        
dataset = pd.read_csv('WW.csv', names=column_names, skipinitialspace=True)      
train_dataset = dataset.sample(frac=0.9,random_state=0)
test_dataset = dataset.drop(train_dataset.index)    
train_labels = train_dataset.iloc[:, 3:].values
test_labels = test_dataset.iloc[:, 3:].values   

print(train_dataset)
print(test_dataset)                         

def build_model():
  model = keras.Sequential([
    keras.layers.Dense(3, activation='relu', input_shape=[len(train_dataset.keys())]),
    keras.layers.Dense(4, activation='relu'),
    keras.layers.Dense(2)
  ])

  optimizer = tf.keras.optimizers.RMSprop(0.001)

  model.compile(loss='mse',
                optimizer=optimizer,
                metrics=['mae', 'mse'])
  return model

model = build_model()
model.summary()

class PrintDot(keras.callbacks.Callback):
  def on_epoch_end(self, epoch, logs):
    if epoch % 100 == 0: print('')
    print('.', end='')

EPOCHS = 5000

early_stop = keras.callbacks.EarlyStopping(monitor='val_loss', patience=500)
history = model.fit(train_dataset, train_labels, epochs=EPOCHS, validation_split = 0.2, verbose=0, callbacks=[early_stop, PrintDot()])
model.save("model.h5")

hist = pd.DataFrame(history.history)
hist['epoch'] = history.epoch
print('\n', hist.tail())

def plot_history(history):
  hist = pd.DataFrame(history.history)
  hist['epoch'] = history.epoch

  plt.figure()
  plt.xlabel('Epoch')
  plt.ylabel('Mean Abs Error [MPG]')
  plt.plot(hist['epoch'], hist['mae'],
           label='Train Error')
  plt.plot(hist['epoch'], hist['val_mae'],
           label = 'Val Error')
  plt.ylim([0,5])
  plt.legend()

  plt.figure()
  plt.xlabel('Epoch')
  plt.ylabel('Mean Square Error [$MPG^2$]')
  plt.plot(hist['epoch'], hist['mse'],
           label='Train Error')
  plt.plot(hist['epoch'], hist['val_mse'],
           label = 'Val Error')
  plt.ylim([0,20])
  plt.legend()
  plt.show()


plot_history(history)

test_predictions = model.predict(test_dataset)
test_dataset['Predicted Nu'], test_dataset['Predicted CD'] = test_predictions[:,0], test_predictions[:,1]
print(test_dataset)

np.savetxt('test_dataset.txt', test_dataset, delimiter=',')

predict = model.predict(train_dataset)
train_dataset['Predicted Nu'], train_dataset['Predicted CD'] = predict[:,0], predict[:,1]
print(train_dataset)

np.savetxt('train_dataset.txt', train_dataset, delimiter=',')

class_names = ['Reynolds Number', 'Blockage Ratio', 'Prandtl Number', 'junk Nusselt Number', 'junk Drag Coefficient']    
all_inputs = pd.read_csv('Predict_Input.csv', names=class_names, skipinitialspace=True)
all_outputs = model.predict(all_inputs)
all_inputs['Predicted Nu'], all_inputs['Predicted CD'] = all_outputs[:,0], all_outputs[:,1]
print(all_inputs)

Answer 1

So to reiterate what you want, you have a dataset with 5 features in total. And you need to use the first three features as inputs and the last two as targets. Here's what needs to changed to achieve that.

import tensorflow as tf
from tensorflow import keras
import numpy as np 
import matplotlib.pyplot as plt 
import pandas as pd 
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
import pickle

column_names = ['Reynolds Number', 'Blockage Ratio', 'Prandtl Number', 'Nusselt Number', 'Drag Coefficient']        
dataset = pd.read_csv('WW.csv', names=column_names, skipinitialspace=True)      
train_dataset = dataset.sample(frac=0.9,random_state=0)
test_dataset = dataset.drop(train_dataset.index)    
train_labels = train_dataset.iloc[:, 3:].values
test_labels = test_dataset.iloc[:, 3:].values   

print(train_dataset)
print(test_dataset)                         

Model building

Your model should have only three features. Therefore, the input_shape should be 3 (not 5). Batch shape will be automatically added by Keras.

# We are setting the input size as (None, 3)
def build_model():
  model = keras.Sequential([
    keras.layers.Dense(3, activation='relu', input_shape=(3,)),
    keras.layers.Dense(4, activation='relu'),
    keras.layers.Dense(2)
  ])

  optimizer = tf.keras.optimizers.RMSprop(0.001)

  model.compile(loss='mse',
                optimizer=optimizer,
                metrics=['mae', 'mse'])
  return model

model = build_model()
model.summary()

class PrintDot(keras.callbacks.Callback):
  def on_epoch_end(self, epoch, logs):
    if epoch % 100 == 0: print('')
    print('.', end='')

Training the model

When training the model you only give the first three features as inputs and the last two features as labels/targets. Now you should see how that helps to solve your problem. Now you can safely predict the two unknown variables by only using the known three features.

So when training as input we only give the first three columns and targets will be the last two.

EPOCHS = 5000

early_stop = keras.callbacks.EarlyStopping(monitor='val_loss', patience=500)

# Note that the input only takes the first three columns
history = model.fit(train_dataset.iloc[:,:3], train_labels, epochs=EPOCHS, validation_split = 0.2, verbose=0, callbacks=[early_stop, PrintDot()])
model.save("model.h5")

Testing time

Same rule goes for the testing and viola! you are not using the unknown two features at the testing time to predict (i.e. we're only using the first three features).

test_predictions = model.predict(test_dataset.iloc[:,:3])
print(test_dataset)
test_dataset['Predicted Nu'], test_dataset['Predicted CD'] = test_predictions[:,0], test_predictions[:,1]
print("\nPredicted\n")
print(test_dataset)